ROT13

Here is where the magnanimous folks that create externals post them (.zip archives only please).

ROT13

Postby codegreen » Thu Oct 17, 2013 2:11 pm

As requested. Syntax is pretty basic:

Code: Select all
put ROT13("Jung'f hc qbp?")

For those interested in such things, the meat of this looks like so:

Code: Select all
void doMyFunction(void)
{
   char     *pIN, *pOUT, c;
   UInt32   i, len;
   
   if (!(par->returnValue = NewHandle(len = GetHandleSize(par->params[0]))))
      Fail("Not enough memory");

   else for (i = 0, pIN = *par->params[0], pOUT = *par->returnValue; i < len; i++)
      *pOUT++ = !isalpha(c = *pIN++) ? c : c + (toupper(c) > 'M' ? -13 : 13);
}

HTH,
-Mark

ROT13.bundle.zip
(4.67 KiB) Downloaded 402 times
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby sctell » Thu Oct 17, 2013 10:40 pm

Just a quick question.

drpitcairn was asking for a version for both PowerPC and Intel.

Does this fit the bill?


Thanks

Terry
sctell
 
Posts: 1134
Joined: Sun Jul 06, 2008 10:41 am

Re: ROT13

Postby codegreen » Thu Oct 17, 2013 11:46 pm

Yes.
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Fri Oct 18, 2013 9:53 am

Thanks for this. Now I am not sure which to use. Should I use this external you made? Or would it be better to use shell commands we have been discussing? I am wondering if an external is more limited in terms of computer OS or something.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Fri Oct 18, 2013 11:31 am

Well even on an SSD (a best-case scenario for disk I/O overhead) you will likely find the XFcn to be literally thousands of times faster than that shell script (which has a lot more overhead).

On the other hand, knowing something about shell scripting (and the vicissitudes of using it from SuperTalk) is good too. Someday it'll save your neck...

8-)
-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Fri Oct 18, 2013 11:44 am

I think what I will do then is to first use your external, see if it suffices for now (as I am ready to release this project to the public). Will also continue to study shell scripts. I have downloaded some info on that and going through it. I was surprised to see how many books and tutorials available. It will be a good thing to know.
I am even thinking maybe I can substitute a major function of this project — which is to search through this text file to find the sentences containing 2 or 3 entered search words in the same sentence then returning them to a field. It works OK but not real fast, taking about 8-10 seconds on my MBPro with an SSD. I imagine slower on other computers. I bet there is a better way to do this search.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby drpitcairn » Fri Oct 18, 2013 1:12 pm

Well good news and bad news. My tester was able to use the project on a PowerPC running Tiger. An old model of course but the project with your new external did run fine, so great. Bad news is that it took over 6 minutes to find a single word in sentences, so obviously not practical for that slow a computer. On mine it would take 5-8 seconds. I had no idea there would be such a difference in time but is obvious on reflection.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Fri Oct 18, 2013 3:18 pm

drpitcairn wrote:I am even thinking maybe I can substitute a major function of this project — which is to search through this text file to find the sentences containing 2 or 3 entered search words in the same sentence then returning them to a field. It works OK but not real fast, taking about 8-10 seconds on my MBPro with an SSD. I imagine slower on other computers. I bet there is a better way to do this search.

Probably. It will likely be much easier if your text is broken down into one sentence per line. That simplifies using tools like grep and perl.

For example:

Code: Select all
get shell(merge("perl -ne 'print if /[[<tString1>]]/ && /[[<tString2>]]/' `[[hfsToPosix(<tHFSPath>)]]`"))

should return all the lines in the file at tHFSPath which contain both tString1 and tString2.

How are you doing this search now, and how is the data organized? Perhaps there's a faster straight SuperTalk solution...

-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Fri Oct 18, 2013 7:44 pm

Mark,
In a way this is a simple project. It accesses a medical text and searches for words occurring together in the same sentence and then extracts those sentences, identifies which medicine they are found under, and lists them in a field. Of course from there various things can be done with the data but this is the heart of it.
Here is a picture of the card that is used for those searches, the words in the top field and the sentences listed where they were found.
skin itch.gif
search card
skin itch.gif (54.41 KiB) Viewed 14050 times

The problem is that the database is quite large, the book it comes from is about 250 pages. So the searching of the entire database takes some time. On my computer just a few seconds, but slower ones, several minutes.

Here is the script that does the searching. I should add that it is more complicated in that the search is for just the start of words rather than the whole word. In the example shown here, I put in the two words "skin" and "itch" to search for. By searching for "itch" and finding all the words that start with these letters, then it also finds "itching", "itched", "itches", etc. This is a very important part of this because of the variation of many of the words.

Code: Select all
on searchNow
    -- GREMDATA IS THE TEXT FILE.
  -- GSEARCHPHRASE IS USED TO TAKE THE VALUE OF LINE CLICKED ON IN RESULTS AND FINDING IN BIGLIST.
  global gremData, gsearchPhrase, theResults, searchTerms, tfieldName
 
  put the short name of cd fld "searchresults" into tfieldName
 
  -- PROCESSES THE SEARCH TERMS
  put cd fld "search" into searchTerms
  if last char of searchTerms is space then delete last char of searchTerms
  put replace(searchTerms, ",","") into searchTerms
  put replace(searchTerms, ";","") into searchTerms
  put replace(searchTerms, ":","") into searchTerms
  put replace(searchTerms, "(","") into searchTerms
  put replace(searchTerms, ")","") into searchTerms
 
  repeat with q = 1 to the num of words in searchTerms
    if first char of word q of searchTerms is quote then
      if last char of word q of searchTerms is not quote then
        answer "Enclose your word with quotes at start and end of word."
        exit script
      end if
    end if
    if first char of word q of searchTerms is "[" then
      if last char of word q of searchTerms is not "]" then
        answer "If you are searching for authors put brackets at start and end of word, like this - [A.]."
        exit script
      end if
    end if
  end repeat
 
  -- MODIFIES RAWDATA FOR SEARCHING. THE DELIMITERS ARE CHANGED SO THAT ONE REMEDY IS PROCESSED AT A TIME.
  put empty into cd fld "searchresults"
  put empty into cd fld "remedynumber"
  set the linedel to "•"  -- CHANGES THE CR THAT INDICATES END OF LINE TO THIS MARKER INSTEAD.
  set the itemdel to cr  -- CONVERTS THE ITEM (A REMEDY DATA) TO ONE LONG PARAGRAPH.
  put empty into theResults
  --  put empty into gsearchPhrase
 
  -- PROGRESSBAR
  put empty into barNumber
  set the minimumValue of cd btn "updateprogress" to 0
  set the maximumValue of cd btn "updateprogress" to the num of lines of gremData
  show cd btn "updateprogress"
 
  -- NOW CHECKS EACH "LINE" OF RAWDATA. A LINE IS THE THE COMPLETE REMEDY DESCRIPTION.
  -- SO CHECKS EACH REMEDY DATA SEQUENTIALLY.
  repeat for each line lineToCheck of gremData
    add 1 to barNumber
    put lineToCheck into thisBigLine  -- PUTS THE SINGLE, YET COMPLETE, REMEDY DATA INTO THE VARIABLE, TREATING IT AS ONE LINE.
   
    put findInStuff(searchTerms,thisBigLine) after theResults  --THE FUNCTION FOR FINDING.
    set the currentValue of cd btn "updateprogress" to barNumber
  end repeat
  if theResults is empty then
    put "Sorry, no results found" into theResults
  end if
  put empty into cd fld "storage"
  lock screen
  put theResults into cd fld "searchResults"
  hide cd btn "updateprogress"
  makeitnice
  unlock screen
end searchNow


The next part of the script which does the word identification is:

Code: Select all
function findInStuff needle, haystack
-- NOW THE SINGLE REMEDY DATA IS MODIFIED BACK TO INDIVIDUAL LINES TO PROCESS.
set the linedel to cr
put the itemdel into oldDel
set the itemdel to tab

put line 1 of haystack into theTitle  -- LINE 1 IS THE NAME OF THE REMEDY.
put empty into theResults
put needle into searchTerms  -- THIS WILL BE ALL THE WORDS TO BE SEARCHED.

-- TAKES EACH LINE OF THE REMEDY UNDER CONSIDERATION AND CHECKS IT FOR THE SEARCH WORDS.
repeat for each line h of haystack
put 0 into showThis
put h into thisLineX
if thisLineX = theTitle then next repeat
if thisLineX starts with "GENERAL INDICATIONS" then next repeat
if thisLineX contains "______" then next repeat
if thisLineX starts with "(" then next repeat
put thisLinex into thisLine

-- TAKES OUT PUNCTUATION FOR EACH LINE.
put replace(thisLine,comma,"") into thisLine
put replace(thisLine,"(","") into thisLine
put replace(thisLine,")","") into thisLine
put replace(thisLine,":","") into thisLine
put replace(thisLine,";","") into thisLine
put replace(thisLine,"!","") into thisLine

-- EACH SEARCH WORD IS LOOKED FOR.
repeat for each word w of searchTerms
put the number of chars of w into charCount  -- COUNTS TO CHARS IN THE SEARCH TERM AND THEN WILL LOOK
-- AT SAME CHAR NUMBER IN EACH WORD BEING SEARCHED. 

-- IF WORD IN QUOTES THEN THIS PART OF SCRIPT IS USED.
if char 1 of w is quote then   
put replace(w,quote,"") into w
if w is in thisLine then
repeat for each word w2 of thisLine
if w = w2 then
add 1 to showThis
exit repeat
end if
end repeat
end if

else

-- IF WORD NOT IN QUOTES THEN THIS PART OF SCRIPT IS USED.
if w is in thisLine then
repeat for each word w2 of thisLine
if w is in w2 then
if char 1 to charCount of w2 is w then -- SEARCHING FROM START OF CHARS TO END, SO CAN CAPTURE LONGER WORDS.
-- IF THE CHARS MATCH, SEARCH TERM AND WORD BEING SEARCHED, THEN THAT LINE IS SELECTED AS A RESULT.
add 1 to showThis
exit repeat
end if
end if
end repeat
end if
end if

end repeat

-- IN SEARCHING EACH LINE WHEN A WORD IS FOUND, THEN THE NUMBER 1 IS ADDED TO SHOWTHIS VARIABLE.
-- IT COULD BE ONLY ONE WORD OF MULTIPLE SEARCH WORDS IS FOUND IN WHICH CASE THE LINE IS NOT SELECTED.
-- THIS NEXT PART CONFIRMS THAT ALL THE SEARCH WORDS OCCUR IN THE SENTENCE UNDER CONSIDERATION.
if showThis is the number of words of searchTerms then
put replace(thisLineX,":"," -") into thisLineX
put theTitle & ":" & tab & thisLineX & cr after theResults
end if
end repeat

-- RETURNS TO ORIGINAL FORMATTING FOR USE BY THE SEARCH NOW SCRIPT.
set the linedel to "•"
set the itemdel to oldDel

return theResults
end findInStuff


The text file that is being processed is the one stored in the userprop, the one we have been working with for text rotation.
Sorry this is so complicated, but I am not sure how else to show it to you.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Sat Oct 19, 2013 4:17 am

Yikes!

Okay, just off the top of my head I would suggest you look over these (completely untested) tweaks to your scripts, and see if anything in there seems useful:

Code: Select all
on searchNow
  -- GREMDATA IS THE TEXT FILE.
  global gremData, theResults
  local searchTerms
 
  -- PROCESS THE SEARCH TERMS
  get lower(stripChars(",;:()!", cd fld "Search"))
  repeat for each word aTerm of it
    if aTerm starts with quote and not (aTerm ends with quote) then
      answer "Enclose your word with quotes at start and end of word."
      exit script
    else if aTerm starts with "[" and not (aTerm ends with "]") then
      answer "To search for authors put brackets around the name, like this - [A.]."
      exit script
    end if
    if aTerm starts with quote then
      put upper(replace(aTerm, quote, "")) & space after searchTerms
    else put aTerm & space after searchTerms
  end repeat
 
  set the linedel to "•"  -- CHANGE THE CR THAT INDICATES END OF LINE TO '•' INSTEAD.
  put empty into cd fld "searchresults"
  put empty into cd fld "remedynumber"
 
  -- PROGRESSBAR
  enum iterations = 0, progressBarBtn = the num of cd btn "updateprogress", ¬
    numRemedies = the num of lines of gremData, progressBarStep = numRemedies div 100
  set the minimumValue of cd btn progressBarBtn to 0
  set the maximumValue of cd btn progressBarBtn to 100
  show cd btn progressBarBtn
 
  -- CHECKS EACH REMEDY DATA SEQUENTIALLY.
  local theResults
  repeat for each line aRemedy of gremData
    put findTerms(searchTerms, aRemedy) after theResults  --THE FUNCTION FOR FINDING.
    add 1 to iterations
    if iterations mod progressBarStep = 0 then set the currentValue of cd btn ¬
      progressBarBtn to the currentValue of cd btn progressBarBtn + 1
  end repeat
 
  if theResults is empty then put "Sorry, no results found" into theResults
  put empty into cd fld "storage"
  lock screen
  put theResults into cd fld "searchResults"
  hide cd btn progressBarBtn
  makeitnice
  unlock screen
end searchNow

function stripChars charList, fromText
  local outStr
  repeat for each char c of fromText
    if c is in charList then next repeat
    put c after outStr
  end repeat
  return outStr
end stripChars

function findTerms searchTerms, remedyData
  enum upperCaseA = charToNum("A"), upperCaseZ = charToNum("Z")
  local theResults

  put line 1 of remedyData into remedyName  -- LINE 1 IS THE NAME OF THE REMEDY.
 
  -- TAKES EACH LINE OF THE REMEDY UNDER CONSIDERATION AND CHECKS IT FOR THE SEARCH WORDS.
  enum hitCount = 0
  repeat for each line aLine of remedyData
    if aLine = remedyName then next repeat
    if aLine starts with "GENERAL INDICATIONS" then next repeat
    if aLine contains "______" then next repeat
    if aLine starts with "(" then next repeat
   
    put stripChars(",;:()!", aLine) into thisLine -- Is this step really necessary?
   
    repeat for each word theTerm of searchTerms
      -- IF SEARCH TERM IS IN UPPERCASE THEN SEARCH FOR AN EXACT MATCH.
      get charToNum(theTerm)
      if it ≥ upperCaseA and it ≤ upperCaseZ then
        -- chunkOffset() is in Xtend, but those last two flags are documented backwards
        if chunkOffset(theTerm, thisLine, 0, cr, false, false, true) > 0 then
          add 1 to hitCount
          exit repeat
        end if
      else if theTerm is in thisLine then -- OTHERWISE 'STARTS WITH' MATCHES ARE OKAY.
        repeat for each word aWord of thisLine
          if aWord starts with theTerm then
            add 1 to hitCount
            exit repeat
          end if
        end repeat
      end if
    end repeat
   
    if hitCount = the number of words of searchTerms
    then put remedyName & ":" & tab & replace(aLine, ":", " -") & cr after theResults
  end repeat
 
  return theResults
end findTerms

Assuming this actually works (which is a big if, since I haven't had my coffee yet) it's probably faster and somewhat more readable than yours (and will be faster yet if that call to stripChars in findTerms turns out to be unneeded). If not, then writing an XFcn equivalent for stripChars() would be an easy way to speed things up a bit.

Note that this version doesn't bang on the progress bar as much as yours. They're surprisingly inefficient (thanks Apple), and you should always try to avoid updating them unnecessarily.

Without your project to drop it into this was impossible to test, so if anything doesn't work (and you can't figure out why) just yell...

If this is still too slow, then obviously rolling the whole search operation into an external would speed things up further.

-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby codegreen » Sat Oct 19, 2013 4:46 am

This should probably improve things a bit further:

Code: Select all
  -- CHECK EACH REMEDY DATA SEQUENTIALLY.
  local theResults
  repeat for each line aRemedy of gremData
    get findTerms(searchTerms, aRemedy)  --THE FUNCTION FOR FINDING.
    if it is not empty then put it after theResults
    add 1 to iterations
    if iterations mod progressBarStep = 0 then set the currentValue of cd btn ¬
      progressBarBtn to the currentValue of cd btn progressBarBtn + 1
  end repeat

HTH,
-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Sat Oct 19, 2013 11:13 am

Mark,
Yikes is right! A lot of lines. I put your script into a project copy to work with and it does not work yet. I found one thing was that there was a global theResult as well as later in the script a local theResult. Clearing that up, it runs but does not yet return any data. So far just the number 00. But I am going through it line by line and using trace and figuring out some of it as I go along.
Processing the search terms looks right (trace), can't evaluate the progressbar fully yet but I think it is working right. I had to change some of the terms from:
Code: Select all
  -- PROGRESSBAR
  enum iterations = 0, progressBarBtn = the num of cd btn "updateprogress", ¬
    numRemedies = the num of lines of gremData, progressBarStep = numRemedies div 100
  set the minimumValue of cd btn progressBarBtn to 0
  set the maximumValue of cd btn progressBarBtn to 100
  show cd btn progressBarBtn

to
Code: Select all
-- PROGRESSBAR
enum iterations = 0, progressBarBtn = the num of cd btn "updateprogress", ¬
numRemedies = the num of lines of gremData, progressBarStep = numRemedies div 100
set the minimumValue of cd btn "updateprogress" to 0
set the maximumValue of cd btn "updateprogress" to 100
show cd btn "updateprogress"

This is a function, right?
Code: Select all
-- PROCESS THE SEARCH TERMS
get lower(stripChars(",;:()!", cd fld "Search"))

and it then access this script?
Code: Select all
function stripChars charList, fromText
local outStr
repeat for each char c of fromText
if c is in charList then next repeat
put c after outStr
end repeat
return outStr
end stripChars

It looks like it is working with trace.
I also see the output of search terms, and they appear correct.

I have no idea what this means:
Code: Select all
enum upperCaseA = charToNum("A"), upperCaseZ = charToNum("Z")

Looks like it is counting up the number of chars in the data line?

You ask:
Code: Select all
put stripChars(",;:()!", aLine) into thisLine -- Is this step really necessary?

and I think maybe it is not. I think I would have to have the script working to be sure but don't see offhand why it is necessary unless it interferes in finding the start of a word. Doesn't seem like it would.

This part is clever. You have tagged the use of quotes in the search field by changing the text to uppercase?
Code: Select all
repeat for each word theTerm of searchTerms
-- IF SEARCH TERM IS IN UPPERCASE THEN SEARCH FOR AN EXACT MATCH.
get charToNum(theTerm)
if it ≥ upperCaseA and it ≤ upperCaseZ then
-- chunkOffset() is in Xtend, but those last two flags are documented backwards
if chunkOffset(theTerm, thisLine, 0, cr, false, false, true) > 0 then
add 1 to hitCount
exit repeat
end if

The rest of this part, just put in above, is processing the search words that are not in quote, right?
Code: Select all
else if theTerm is in thisLine then -- OTHERWISE 'STARTS WITH' MATCHES ARE OKAY.
repeat for each word aWord of thisLine
if aWord starts with theTerm then
add 1 to hitCount
exit repeat
end if
end repeat
end if
end repeat

Then, if all the search terms are found:
Code: Select all
if hitCount = the number of words of searchTerms
then put remedyName & ":" & tab & replace(aLine, ":", " -") & cr after theResults
end repeat


Am I getting a reasonable sense of it? I still don't have it working yet.
I put in this as the text to access, gremData, to make it simple:
Abies Canadensis
NAME: HEMLOCK SPRUCE

and then I search for the word "name" and get this result:
name.png
name.png (14.23 KiB) Viewed 14042 times

so not right and where I am stuck at the moment.

My wife an artist here in Sedona, AZ, and is one of the featured artists for a national event starting today in which all week they paint landscapes outdoors. Lots of people come, sales, auctions, parties, etc. So I am off to help her for the rest of the day, but will check for your sage advice later.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Sat Oct 19, 2013 1:04 pm

I have no idea what this means:

Code: Select all
enum upperCaseA = charToNum("A"), upperCaseZ = charToNum("Z")

It just sets up two variables to compare a character's ASCII value against to determine if it's uppercase or not (since in SuperTalk "a" = "A", this is one way to tell them apart).

This part is clever. You have tagged the use of quotes in the search field by changing the text to uppercase?

Yes, that way you don't have to reformat the search terms each time you parse another remedy.

Here's another shortcut that might help:

Code: Select all
    repeat for each word theTerm of searchTerms
      -- IF SEARCH TERM IS IN UPPERCASE THEN SEARCH FOR AN EXACT MATCH.
      get charToNum(theTerm)
      if it ≥ upperCaseA and it ≤ upperCaseZ then
        if chunkOffset(theTerm, thisLine, 0, cr, false, false, true) > 0 then
          add 1 to hitCount
          exit repeat
        end if
      else if space & theTerm is in space & thisLine then -- FASTER 'STARTS WITH' TEST
        add 1 to hitCount
        exit repeat
      end if
    end repeat

-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby codegreen » Sat Oct 19, 2013 2:32 pm

Okay, I think I see the problem. Try this:

Code: Select all
on searchNow
  global gremData, theResults
  local searchTerms
 
  put empty into cd fld "searchresults"
  put empty into cd fld "remedynumber"
 
  -- PROCESS THE SEARCH TERMS
  get lower(stripChars(".,;:()!", cd fld "Search"))
  repeat for each word aTerm of it
    if aTerm starts with quote and not (aTerm ends with quote) then
      answer "Enclose your word with quotes at start and end of word."
      exit script
    else if aTerm starts with "[" and not (aTerm ends with "]") then
      answer "To search for authors put brackets around the name, like this - [A.]."
      exit script
    end if
    if aTerm starts with quote then
      put upper(replace(aTerm, quote, "")) & space after searchTerms
    else put aTerm & space after searchTerms
  end repeat
 
  delete last char of searchTerms
  if searchTerms is empty then
    answer "Please enter terms to search for."
    exit script
  end if
 
  set the linedel to "•"  -- CHANGE THE CR THAT INDICATES END OF LINE TO '•' INSTEAD.
 
  -- PROGRESSBAR
  enum iterations = 0, numRemedies = the num of lines of gremData, ¬
    numSteps = min(numRemedies, 100), progressBarBtn = the num of ¬
    cd btn "updateprogress", progressBarStep = numRemedies div numSteps
  set the minimumValue of cd btn progressBarBtn to 0
  set the currentValue of cd btn progressBarBtn to 0
  set the maximumValue of cd btn progressBarBtn to numSteps
  show cd btn progressBarBtn
 
  -- CHECK EACH REMEDY DATA SEQUENTIALLY.
  put empty into theResults
  repeat for each line aRemedy of gremData
    get findTerms(searchTerms, aRemedy)  --THE FUNCTION FOR FINDING.
    if it is not empty then put it after theResults
    add 1 to iterations
    if iterations mod progressBarStep = 0 then set the currentValue of cd btn ¬
      progressBarBtn to the currentValue of cd btn progressBarBtn + 1
  end repeat
 
  if theResults is empty then put "Sorry, no results found" into theResults
  put empty into cd fld "storage"
  lock screen
  put theResults into cd fld "searchResults"
  hide cd btn progressBarBtn
  makeitnice
  unlock screen
end searchNow

function stripChars charList, fromText
  local outStr
  repeat for each char c of fromText
    if c is in charList then next repeat
    put c after outStr
  end repeat
  return outStr
end stripChars

function findTerms searchTerms, remedyData
  enum upperCaseA = charToNum("A"), upperCaseZ = charToNum("Z")
 
  put empty into theResults
  put line 1 of remedyData into remedyName  -- LINE 1 IS THE NAME OF THE REMEDY.
 
  repeat for each line aLine of remedyData
    enum hitCount = 0
    if aLine = remedyName then next repeat
    if aLine starts with "GENERAL INDICATIONS" then next repeat
    if aLine contains "______" then next repeat
    if aLine starts with "(" then next repeat
   
    put stripChars(".,;:()!", aLine) into thisLine
   
    repeat for each word theTerm of searchTerms
      get charToNum(theTerm)
      if it ≥ upperCaseA and it ≤ upperCaseZ then
        if char 1 of chunkOffset(lower(theTerm), lower(thisLine), 0, ¬
          cr & space, false, false, true) ≠ 0 then
          add 1 to hitCount
        end if
      else if space & theTerm is in space & thisLine then add 1 to hitCount
      else exit repeat
    end repeat
   
    if hitCount = the number of words of searchTerms
    then put remedyName & ":" & tab & replace(aLine, ":", " -") & cr after theResults
  end repeat
 
  return theResults
end findTerms

HTH,
-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Sat Oct 19, 2013 6:07 pm

Thanks. Just got in late. I will look this over tomorrow, too tired now. I did see that this part is the as it was before and have a question.
Code: Select all
  -- PROGRESSBAR
  enum iterations = 0, numRemedies = the num of lines of gremData, ¬
    numSteps = min(numRemedies, 100), progressBarBtn = the num of ¬
    cd btn "updateprogress", progressBarStep = numRemedies div numSteps
  set the minimumValue of cd btn progressBarBtn to 0
  set the currentValue of cd btn progressBarBtn to 0
  set the maximumValue of cd btn progressBarBtn to numSteps
  show cd btn progressBarBtn

The existing button has the name cd btn "updateprogress". So I change the minimumvalue and maxvalue lines to show that name? Also the show btn line? Or is there something I don't understand here?
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Sat Oct 19, 2013 6:51 pm

Names are the slowest form of object descriptor to parse, while numbers are the fastest. We're going to be referring to the object repeatedly, but we only need to do the expensive version of the lookup once. You could just hard-code the number of your progress bar, but that would be unnecessarily fragile...
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Sun Oct 20, 2013 12:37 pm

I have been working this morning on the project, got the script to work.
I had to add this line in, from the prior script on my original project but once I did, then it functioned.

set the linedel to "•" -- CHANGE THE CR THAT INDICATES END OF LINE TO '•' INSTEAD.
set the itemdel to cr -- CONVERTS THE ITEM (A REMEDY DATA) TO ONE LONG PARAGRAPH.

Then I timed the script and compared to my original script. Interestingly the new script was quite a bit slower.
Searching for the word "cough" took 3 sec in original, 15 sec in new script. Final output was 474 lines so a very large output.
Searching for "blood" took 3 sec vs. 15 sec. in new script, output was 309 lines.
Next I commented out the lines that operate the progressbar and there was little difference, new script now took 14 sec to find "blood." So made that part (progress bar) active again.

Then worked with this script:
Code: Select all
function findTerms searchTerms, remedyData
  enum upperCaseA = charToNum("A"), upperCaseZ = charToNum("Z")
  put empty into theResults
  put line 1 of remedyData into remedyName  -- LINE 1 IS THE NAME OF THE REMEDY.
  repeat for each line aLine of remedyData
    enum hitCount = 0
    if aLine = remedyName then next repeat
    if aLine starts with "GENERAL INDICATIONS" then next repeat
    if aLine contains "______" then next repeat
    if aLine starts with "(" then next repeat
    --    put stripChars(".,;:()!", aLine) into thisLine
    put replace(aLine,"(","") into thisLine
    repeat for each word theTerm of searchTerms
      get charToNum(theTerm)
      if it ≥ upperCaseA and it ≤ upperCaseZ then
        if char 1 of chunkOffset(lower(theTerm), lower(thisLine), 0, ¬
          cr & space, false, false, true) ≠ 0 then
          add 1 to hitCount
        end if
      else if space & theTerm is in space & thisLine then add 1 to hitCount
      else exit repeat
    end repeat
    if hitCount = the number of words of searchTerms
    then put remedyName & ":" & tab & replace(aLine, ":", " -") & cr after theResults
  end repeat
  return theResults
end findTerms

I commented out this one line:
-- put stripChars(".,;:()!", aLine) into thisLine
and the find time for the script dropped from 15 sec to 2 sec. Big difference.
Working with it I realized that the only char I really needed to have out was the "(" character, so I put this line in to handle it:
put replace(aLine,"(","") into thisLine
and checking time again, running the search for the word "cough" (474 lines found) took only 2 sec, so that replace line did not seem to slow things that I could tell. So very encouraging progress.

Then I tested for words that quotes around them and find a problem with that.
In the script I put in above, there is an issue with this part:
repeat for each word theTerm of searchTerms
get charToNum(theTerm)
if it ≥ upperCaseA and it ≤ upperCaseZ then
if char 1 of chunkOffset(lower(theTerm), lower(thisLine), 0, ¬
cr & space, false, false, true) ≠ 0 then
add 1 to hitCount
end if
else if space & theTerm is in space & thisLine then add 1 to hitCount
else exit repeat
end repeat

What comes up is Supertalk error 41, never heard of that function name. So I assume there is a function it can't find.
I did a search for chunkOffset and do not find this function so something we have to add?
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Sun Oct 20, 2013 8:37 pm

I commented out this one line:
-- put stripChars(".,;:()!", aLine) into thisLine
and the find time for the script dropped from 15 sec to 2 sec. Big difference.

Yeah, I pretty much expected that. Adding the overhead of a script function call to that word-parsing loop was bound to slow things down substantially, but it also isolated what I suspected was your single biggest performance hit in a separate module that could easily be turned into an external.

stripChars.bundle.zip
(4.58 KiB) Downloaded 376 times

I had to add this line in, from the prior script on my original project but once I did, then it functioned.

set the linedel to "•" -- CHANGE THE CR THAT INDICATES END OF LINE TO '•' INSTEAD.
set the itemdel to cr -- CONVERTS THE ITEM (A REMEDY DATA) TO ONE LONG PARAGRAPH.

What version of SC are you using? The skeleton of that script seems to work fine here in 4.73 without that change, so I'm guessing it's one which predates localDelimiters?

I did a search for chunkOffset and do not find this function so something we have to add?

chunkOffset() is from Xtend (see the SuperCard Help project).

So how many remedies and lines of data are you searching?

-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Mon Oct 21, 2013 10:19 am

I have been working on this all morning. I am leaning a lot but sometimes my head starts to swim!

I installed the external and tested it and speed is good.

Regarding this line:
set the itemdel to cr -- CONVERTS THE ITEM (A REMEDY DATA) TO ONE LONG PARAGRAPH.
I commented it out and ran the script and it worked just the same. So I have no explanation as to why adding it in before made the script work. Must have been my error. Anyway, running now without it.

Thanks for the info on chunckOffSet. I did not know about and I can see its usefulness.

I am using SC 4.7.3.

The text file I am accessing as my database has 17, 811 lines as of now and I am adding to it still. Might get up to 19,000 before I am done.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Mon Oct 21, 2013 1:18 pm

Just fer grins, would you do me a favor and download this project, double-click it, paste in your database (into the upper of the two scrolling fields, replacing the dummy data that's there now), and try a few searches?

Thanks,
-Mark

findTerms.sp45.zip
(9.65 KiB) Downloaded 367 times
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Mon Oct 21, 2013 1:31 pm

Glad to do it but it appears to need a password to open.
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Mon Oct 21, 2013 1:39 pm

It's a SuperCard Player project, and should launch the SC Player when double-clicked (provided you've installed it -- if not, the installer should be in /Applications/SuperCard 4.73/Delivery Tools/). You only need the password to open it with the regular SC runtime engine...

-Mark
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Mon Oct 21, 2013 2:16 pm

Very interesting. I have never dealt with the player before, but got it going. Problem is that I was able to manually drag across the text (could not use command/A) and delete the text in the field. However, cannot paste into the field from my clipboard. Don't know why, just won't go in.

Would it be easier if I sent you my project?
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Re: ROT13

Postby codegreen » Mon Oct 21, 2013 2:40 pm

D-OH! Okay, try this one...

findTerms.sp45 2.zip
(10.08 KiB) Downloaded 356 times
codegreen
 
Posts: 1540
Joined: Mon Jul 14, 2008 11:03 pm

Re: ROT13

Postby drpitcairn » Mon Oct 21, 2013 2:59 pm

Wow! Search instantaneous. The return message is 5 ticks, 6 ticks for even large searches. What's the trick?
User avatar
drpitcairn
 
Posts: 1023
Joined: Thu Apr 02, 2009 9:39 am
Location: Sedona, Arizona

Next

Return to XCmds, XFcns, and XRtns

Who is online

Users browsing this forum: No registered users and 1 guest

cron