Jump to content


  • Content Count

  • Joined

  • Last visited

  • Days Won


Everything posted by iwato

  1. Yes and know. Please match the following results against the included code. RESULTS listItem: Array [ "subject-verb pairs", 157 ] listItem: Array [ "subject-verb agreement", 157 ] listItem: Array [ "&quotperson and number&quot", 471 ] listItem: Array [ "person", 157 ] listItem: Array [ "´person and number", 157 ] listItem: Array [ "person and happiness", 314 ] listItem: Array [ "´person and happiness", 157 ] listItem: Array [ 141, "الدواعيمنفكرةقرامركابتِف" ] The CODE $.each(jsonData, function(key, object) { var searchItem = {}; var strippedStr = ''; var agex = new RegExp(/[\u0600-\u06ff]|[\u0750-\u077f]|[\ufb50-\ufbc1]|[\ufbd3-\ufd3f]|[\ufd50-\ufd8f]|[\ufd92-\ufdc7]|[\ufe70-\ufefc]|[\uFDF0-\uFDFD]/g); var pungex = new RegExp(/(&#\d{3};)|(&\s{4};)/g); var pregex = new RegExp(/["\'\;]/g); $.each(object, function(name, value) { if (name === 'searchKeyword') { var nakedArabic = []; if (agex.test(value)) { nakedArabic = value.match(agex); searchItem.target = nakedArabic.join(''); } else { var strippedStr = value.replace(pungex, ''); strippedStr = strippedStr.replace(pregex, ''); searchItem.target = strippedStr; } } if (name === 'searchCategory') { searchItem.category = value; } if (name === 'searchResultsCount') { searchItem.count = value; } }); searchItems.push(searchItem); }); Roddy
  2. Yes, the agex variable to which the .test() function is applied contains a regular expression. var agex = new RegExp( /[\u0600-\u06ff] |[\u0750-\u077f] |[\ufb50-\ufbc1] |[\ufbd3-\ufd3f] |[\ufd50-\ufd8f] |[\ufd92-\ufdc7] |[\ufe70-\ufefc] |[\uFDF0-\uFDFD]/g ); This expression is supposed to identify all characters written in Arabic script including numbers, punctuation, and various diacritical markings. I can do the same for double-byte Japanese as well (not shown below). This same procedure cannot be used for ASCII, however. Indeed, I am finding it difficult to remove HTML entities like ĵ and ". When a visitor enters a search term, Matomo appears to use something akin to urlencode( ) before anything is entered into its own database. When I pull it out I am left with encoded HTML entities that I, generally speaking, do not want (neither encoded, nor decoded). This said, an apostrophe between words or letters can be meaningful. Consider for example the following for phrases "my friend's idea", 'my friend's idea', "my friends' ideas", and 'my friends' ideas'. Now, all of the single and double quotation marks are encoded identically, but only the following two are desired friend's friends' ideas My gosh I cannot even write the REGEX to successfully eliminate ĵ and &quot. Compare the following list items with the Arabic item at the bottom. listItem: Array [ "´person and number&#039", 157 ]jquery-1.11.3.min.js%20line%202%20%3E%20eval:107:9 listItem: Array [ "person and happiness", 157 ]jquery-1.11.3.min.js%20line%202%20%3E%20eval:107:9 listItem: Array [ "´person and happiness&#039", 157 ]jquery-1.11.3.min.js%20line%202%20%3E%20eval:107:9 listItem: Array [ "&#039person and happiness&#039", 157 ]jquery-1.11.3.min.js%20line%202%20%3E%20eval:107:9 listItem: Array [ 141, "الدواعيمنفكرةقرامركابتِف" ] Yes, the Arabic list item does not fall in the proper order, but at least it comes out clean. The following REGEX that I wrote myself simply does not work. var pungex = new RegExp(/(&#\d{3};)|(&\s{4};)/g); The CODE if (name === 'searchKeyword') { var nakedArabic = []; var arabicText = ''; if (agex.test(value)) { nakedArabic = value.match(agex); searchItem.target = nakedArabic.join(''); } else { var strippedStr = value.replace(pungex,''); searchItem.target = strippedStr; } }
  3. OK. I will try to make the problem easier to understand. In order to do so, please examine carefully the following code snippet. $.each(sourceObj, function(searchPhrase, searchCount) { listItem = [searchPhrase, searchCount]; if (agex.test(searchPhrase)) { listItem = listItem.reverse(); } list.push(listItem); }); The phrase agex.test(searchPhrase) tests whether the value of searchPhrase is in Arabic. The result appears something like what follows: listItem: Array [ "構成", 157 ] listItem: Array [ "逆転分析", 157 ] listItem: Array [ "langue sécondaire", 157 ] listItem: Array [ "système éducatif", 157 ] listItem: Array [ "monde académique", 157 ] listItem: Array [ "Rundbrief", 157 ] listItem: Array [ "Sieben Tore", 157 ] listItem: Array [ "Socratic method", 157 ] listItem: Array [ "podcast", 157 ] listItem: Array [ "subject-verb", 157 ] listItem: Array [ "subject-verb pairs", 157 ] listItem: Array [ "subject-verb agreement", 157 ] listItem: Array [ 141, "الدواعيمنفكرةقرامركابتِف" ] If I do not reverse the order of the item, the corresponding listItem is returned as follows: listItem: Array [ "141, "الدواعيمنفكرةقرامركابتِف ] Either way the phrase and count are reverse in the final list and Wordcloud2 fails. When the value of searchPhrase is Arabic agex.test(searchPhrase) returns true as expected. Roddy
  4. BACKGROUND: Not only has my website been spammed, but the spam appears to be of a special sort. It appears that someone has discovered a way to confuse Wordcloud2's interpretation of the input data. My reason for believing this is my ability to make Wordcloud2 work with data obtained from within different temporal ranges of the same data field. You can see clearly from the www.grammarcaptive.com mainpage under Visitor Profile/Word Clouds that under normal conditions Wordcloud2 works with all languages. When I open the temporal range to include the spam, however, WordCloud2 fails to render properly. In order to make Wordcloud2 function properly it is necessary to create a List. In the absence of the spam the items of this list appear in the following order. targetList: الدواعي من فكرة قرامر كابتِف,1,構成,1,逆転分析,1,langue sécondaire,1,système éducatif,1,monde académique,1,Rundbrief,1,Sieben Tore,1,Socratic method,1,podcast,1,subject-verb,1,subject-verb pairs,1,subject-verb agreement,1,person and number,3,person,1,´person and number,1,person and happiness,2,´person and happiness,1 In the presence of the spam the list appears as follows: targetList: 構成,157,逆転分析,157,langue sécondaire,157,système éducatif,157,monde académique,157,Rundbrief,157,Sieben Tore,157,Socratic method,157,podcast,157,subject-verb,157,subject-verb pairs,157,subject-verb agreement,157,&quotperson and number&quot,471,person,157,´person and number&#039,157,person and happiness,157,´person and happiness&#039,157,&#039person and happiness&#039,157,&#039&#039الدواعي من فكرة قرامر كابتِف,141 Notice the appearance of the Arabic entries in the two lists. In the first instance the Arabic appears first. In the second instance it appears last. In the first instance the count appears with no HTML entity, in the second it appears with two. QUESTION ONE: How would you go about cleansing the data of the &quot, &#039, and &quotperson before the list is created? It appears to require some sort of REGEX expression. QUESTION TWO: Where would you cleanse the data? Before entry, or after retrieval? Caution: Cleaning the data upon entry would likely destroy the ability to search phrases. Roddy
  5. I am using it in this way: to visualize recorded frequencies for identical search terms, and for recorded frequencies for discovered matches for those terms. I could white list registered members, but even these could be abusive. I have also been thinking of implementing a no-robot captcha clicker. For this would require only a simple click before initiating a search. I have read, however, that even these will only constrain the number of abusers -- not eliminate them. Currently, Matomo collects everything. Whereupon I extract the data from specific fields, and store then in another database. Whereupon visitors call up the data on demand. I suppose I could write an algorithm that would examine the redundancy and variety of searches from a single user over time. My gosh. Must I reinvent Google all over again? This is why I installed Matomo in the first place. In order to avoid having to reinvent the wheel. My goal is not to outguess my visitors; rather, it is to learn from and teach them. How about dual fields in the same database: one temporary and one permanent? Create an algorithm that scans the temporary field and fill from the result the field that is used to create the respective word clouds? Still, I would need guidelines for creating the algorithm's parameters. Roddy
  6. The term word cloud is very well defined, if you open to the suggested menu item, for there you can see the currently perverted result. More importantly, how does one go about validating terms that user's make up? Indeed, the whole purpose of the search engines and accompanying word clouds is to discover what my visitor's want and what they are able to find. For, in this way I can better serve them, and they can better serve themselves. Specifically, Visitor enter a keyword or phrase in the search box. A search is made for the keyword or phrase and all fields of a certain kind that contain that word or phrase are returned. A count is made of the number of matched rows within the relevant fields. The search keyword or phrase, the count, and an name for the indexed fields is sent to the local Matomo database The values for the above three variables are then parsed and two word clouds are generated: one for the number of times that a particular keyword or phrase has been made; and one for the number of times that a match has been found. Where would the validation take place in the above scheme of things? And, what form are you suggesting that it would take? The only truly secure way, of which I can think, to insure that spamming does not take place is to deny visitors who are not registered member from use of the search procedures. This, however, appears extreme. Roddy
  7. BACKGROUND: Many months ago I created three custom search engines and two word clouds. These latter are used to track visitor search interest and Grammar Captive's ability to match it. Unfortunately, someone is now abusing my search engines and filling those fields of my database set aside to track search input. The result of this malicious effort is word clouds that no longer perform their task. While investigating the mess I have realized two important short-comings of my search engines that I must address: The Sanitization/Filtering of Search Input -- In order to produce meaningful word clouds I must clean my search input of non-meaningful characters such as ', ", /, [, (, &quot, etc. As Further, as my search engines are capable of handling multilingual input, I must be able to handle unwanted characters in a variety of languages. Span Prevention - I must be able to prevent visitors from abusing my search engines and thus destroying Grammar Captive's ability to track and analyze true visitor interest. I do not see myself as resolving these two problems easily and would be grateful for any advice that would push in in the direction of successful resolution. Roddy PS. If you would like to view the damage that has occurred, go to the Grammar Captive website and examine the word clouds found under Visitor Profile/Word Clouds in the navigation bar.
  8. Thank you, JMRKER, for your input. The problem was finally resolved using the following .filter( ) function. var uniqVals = visitData.filter((obj, index, self) => index === self.findIndex((other) => ( other.visitorID === obj.visitorID && other.fullDate === obj.fullDate )) ); Roddy
  9. The Data var visitData = [ {'visitorID':'6b77f74d969f0254','fullDate':'2019/01/01'}, {'visitorID':'6b77f74d969f0254','fullDate':'2019/01/01'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/03'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/03'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/03'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/03'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/04'}, {'visitorID':'470b11ecb96f51a8','fullDate':'2019/01/04'}, {'visitorID':'470b11ecb96f51a8','fullDate':'2019/01/05'}, {'visitorID':'7783172ce39cd663','fullDate':'2019/01/05'}, {'visitorID':'1834494d8537012d','fullDate':'2019/01/06'}, {'visitorID':'3321037b66164959','fullDate':'2019/01/06'} ]; Alright, I have played with the map function and discovered that the following code displays exactly what I want -- a subset of unique objects. Simply I do not know how to convert it into anything useful. The Code function getUniqAnalysis(inputArr, comp1, comp2) { const unique = inputArr.map(function(e) { console.log('e[comp1]: ' + e[comp1] + ' e[comp2]: ' + e[comp2]); }); } Something Useful var somethingUseful = [ {'visitorID':'6b77f74d969f0254','fullDate':'2019/01/01'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/03'}, {'visitorID':'8305d1d7c98cfe07','fullDate':'2019/01/04'}, {'visitorID':'470b11ecb96f51a8','fullDate':'2019/01/04'}, {'visitorID':'470b11ecb96f51a8','fullDate':'2019/01/05'}, {'visitorID':'7783172ce39cd663','fullDate':'2019/01/05'}, {'visitorID':'1834494d8537012d','fullDate':'2019/01/06'}, {'visitorID':'3321037b66164959','fullDate':'2019/01/06'} ]; Please advise. Roddy
  10. Thank you, SVG, for your explanation of the one property-value pair reduction. My transformation of the one property-value pair reduction into a dual property-value pair reduction works for objects with three properties, but not two. Can you explain why? Roddy
  11. I have erred. What worked for objects with three properties does not work for objects with two properties. I had hoped that my modification would have rendered the following transformation, but it does not. Input Array Object { visitorID: "6b77f74d969f0254", fullDate: "2019/01/01" } Object { visitorID: "6b77f74d969f0254", fullDate: "2019/01/01" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/03" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/03" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/03" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/03" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/04" } Object { visitorID: "470b11ecb96f51a8", fullDate: "2019/01/04" } Object { visitorID: "470b11ecb96f51a8", fullDate: "2019/01/05" } Object { visitorID: "7783172ce39cd663", fullDate: "2019/01/05" } Output Array Object { visitorID: "6b77f74d969f0254", fullDate: "2019/01/01" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/03" } Object { visitorID: "8305d1d7c98cfe07", fullDate: "2019/01/04" } Object { visitorID: "470b11ecb96f51a8", fullDate: "2019/01/04" } Object { visitorID: "470b11ecb96f51a8", fullDate: "2019/01/05" } Object { visitorID: "7783172ce39cd663", fullDate: "2019/01/05" } I expected the above transformation base on the below transformation var dataArr = [ {"name":"Steven Smith","country":"England","age":35}, {"name":"Hannah Reed","country":"Scottland","age":23}, {"name":"Steven Smith","country":"England","age":35}, {"name":"Robert Landley","country":"England","age":84}, {"name":"Steven Smith","country":"England","age":35}, {"name":"Robert Landley","country":"England","age":56} ]; var output = [ {"name":"Steven Smith","country":"England","age":35}, {"name":"Hannah Reed","country":"Scottland","age":23}, {"name":"Robert Landley","country":"England","age":84}, {"name":"Robert Landley","country":"England","age":56} ]; In both cases I used two variables for comparison. In the first case, however, there were only two variables to compare. In the second there was a third variable for which no matching was intended. Alas, where have I gone wrong? Roddy
  12. BACKGROUND: I recently discovered the following piece of code on StackOverflow,. Although it works for a single property comparison, I do not understand it well enough to expand it to cope with multiple properties. function getUnique(inputArr, comp) { const unique = inputArr.map(e => e[comp]).map((e, i, final) => final.indexOf(e) === i && i).filter(e => inputArr[e]).map(e => inputArr[e]); return unique; } I would like to create a function that would permit the comparison of multiple values on the order of function getUnique(inputArr, comp1 [, comp2[, comp3 [...]]]){...} Any suggestions? At minimum could you help with a better understanding of how it does what it does. It maps and filters, but in a manner that I do not comprehend easily enough to manipulate. Roddy PS. A little experimentation came up with the following solution. function getUnique(inputArr, comp1, comp2) { const unique = inputArr.map(e => e[comp1] && e[comp2]) .map((e, i, final) => final.indexOf(e) === i && i) .filter(e => inputArr[e]) .map(e => inputArr[e]); return unique; } It achieves my goal, but a little more understanding is still requested.
  13. Dsonesuk! Have you given up? No new ideas? Roddy
  14. This produces the best, but not the desired, result. It is fully responsive on both my MacBook and iPhone, except for on the iPhone one must scroll after the page opens and after the position of the phone has changed. Upon scrolling the map fills its container as it should. If I do not resize the container the map over extends it, HTML <div id='wm_container'> ... OTHER STUFF ... <div id="vmap"></div> </div><!-- end div#wm_container --> CSS #worldmap_div #wm_container { background-color: #ffffff; margin-top: 3em; margin-bottom: 1em; } #worldmap_div #wm_container #vmap { width: 513px; height: 342px; } JAVASCRIPT onResize: function() { var containerWidth = $(this).parent().width(), containerHeight = (containerWidth * (2/3)); console.log('Parent: ' + $(this).parent().attr('id')); $(this).css({ 'width': containerWidth, 'height': containerHeight }); $(window).find('svg').attr('viewbox', '0 0 513 342'); console.log('Map Size: ' + containerWidth + 'x' + containerHeight); } Roddy
  15. Dsonesuk. It makes no difference how the viewbox width and height values are set; the result is both similar and different. Important is that the map image does not properly fill its container until the visitor as scrolled. I had tried fixing the value as you suggested earlier, but did not tell. My apologies. By the way, I have now tested it on another, non-Apple iPhone. The problem is the same. I conclude therefore that the issues lies with the Javascript. If I have understood the problem correctly. the onResize callback function of the object created by the JQVMaps vectorMap() function responds only responds to container resizing triggered by changes in the browser itself, not the viewport per se. I have concluded this for two reasons: one, the behavior is excellent in my FireFox browser on my MacBook without any scrolling; and two, when using the viewbox attribute, both the horizontal and vertical positions of a smartphone viewport require scrolling -- visitor induced, direct browser interaction. Do you agree with this assessment? Roddy
  16. Oops! Thank you for the correction. I have been away from jQuery for several months ... Ha! Ha! Ha! In any case it makes no difference. Proper alignment only takes place with an iPhone scroll. I have not tested it on Android. onResize: function() { var containerWidth = $(this).parent().width(), containerHeight = (containerWidth * (2/3)); console.log('Parent: ' + $(this).parent().attr('id')); $(this).css({ 'width': containerWidth, 'height': containerHeight }).find('svg').attr('viewbox', "0 0 " + containerWidth + " " + containerHeight); console.log('Map Size: ' + containerWidth + 'x' + containerHeight); }, Here is the link. In the navigation bar look for Visitor Profile / World Map. Roddy
  17. So, I tried this the following. onResize: function() { var containerWidth = $(this).parent().width(), containerHeight = (containerWidth * (2/3)); console.log('Parent: ' + $(this).parent().attr('id')); $(this).css({ 'width': containerWidth, 'height': containerHeight }).find('svg').css('viewbox', "0 0 " + containerWidth + " " + containerHeight); console.log('Map Size: ' + containerWidth + 'x' + containerHeight); } RESULT: It had no effect on my browser responsiveness (a good thing), but it worsened the iPhone experience (bad thing). In any case, it did not solve the problem. In the vertical iPhone position the map is now too large (before it was just right), and in the horizontal position the map is too small (just as before). Upon scrolling both images fill the container div properly. Any more suggestions? Roddy
  18. Hi, Dsonesuk! Truly I do not understand how I could incorporate what you are suggesting in the context of the problem. Surely you understand that the HTML that hosts the SVG object is pulled into an already formatted HTML page with AJAX in the form of a set of nested div tags that have their own style sheet and no document heading. Roddy
  19. The problem appears to be that the a change in the size of the viewport is not automatically registered as a change in the size of container. Once the page is scrolled the map fills the viewport and container as it should. Roddy
  20. I have never used console.log on an iPhone before and have no idea where to look for such information. Else, I would have looked long ago. Roddy
  21. Hi Dsonesuk! It appears not to make a difference. The width(), innerWidth(), and outerWidth() functions all yield the same result on the iPhone. What is unusual is both the placement and size of the map when the iPhone is placed on its side. For, rather than filling the center of the viewport, as it does when the iPhone is vertically positioned, it appears in the upper left-hand corner and retains the size that it had when the iPhone was horizontal. Roddy
  22. Thank you, Ingolme! Almost there. $('#vmap').vectorMap({ map: 'world_en', enableZoom: true, showTooltip: true, onResize: function() { var containerWidth = $(this).parent().width(), containerHeight = (containerWidth * (2/3)); $(this).css({ 'width': containerWidth, 'height': containerHeight }).find('svg').css({ 'width': containerWidth, 'height': containerHeight }); console.log('Map Size: ' + containerWidth + 'x' + containerHeight); } }); The above works well in a Firefox browser, but only partially on my iPhone. Any more suggestions? Roddy
  23. SoOK. So, I added the following onResize() function, but the error message remains the same and there is no change in the imagery. $('#vmap').vectorMap({ map: 'world_en', enableZoom: true, showTooltip: true, onResize: function() { var containerWidth = $('#wm_container').width, containerHeight = (containerWidth / 1.4); $(this).find('svg').css({ 'width': containerWidth, 'height': containerHeight }); console.log('Map Size: ' + containerWidth + 'x' + containerHeight); } }); Roddy
  24. Hello, JSG. Please read my note to Dsonesuk. In regard to the console.log() statement (see below) you would agree then, that the onResize() function provided as an option to the vectorMap() function is unnecessary to the resizing of the document. For, all it appears to do is tell the user what the result of the resize is, when, and if, it actually worked. onResize: function (element, width, height) { console.log('Map Size: ' + width + 'x' + height); } Roddy
  • Create New...