3.0.0 Search

[rejetto]

I think he was asking how many results you would get searching for "*"

[TSG]

It took only 7 seconds to complete.

The problem with slow searches might be template related.

1. Did your HFS already have the search cached? I find it quicker the second time round.

2. Impossible that it is template related. Unless it is HFS struggling to generate a html page to send to the user. If that is the case, templates like live and terayon will take longer than tog, there is a lot more characters/table row in those 2 templates.

Foggy's results are the same as mine. GE, Foggy and I all basically hammered each others HFS for about half an hr to test this feature, it is definitely CPU related. It took me about 4 minutes to recursively show results of my entire PC... close to 200k files it think. There is no way to reduce this, it is the reason why servers need a lot of processing power.

We need to set up some controlled tests. I have a lot of time on my hands but my internet is capped. Maverick if you use msn add my email that is on the forum. I think we should all run controlled tests with different templates and print out results, Mythbusters style.

[Foggy]

I was more or less asking about how many files hfs had to search/list in the recursive search/list.

Some controlled tests are good idea. after this week I'll have plenty of time on my hands, yay school holidays.

[maverick]

If you are going to do any testing use a common template - the best I can suggest is the default "plain jane" template and everybody has it.  If there is a huge difference in results between the default template and a customized template, the problem might be somewhere in the customized template. 

I know with my customized template, the same mp3 search takes a few seconds longer because some of my mp3 files include the mp3player.  I know this and can account for it. 

I just ran the same test on the PII-300mhz machine and this time it took 9 seconds with the same 749 files 2.37 GB search results.  This answers TSG caching question.  Sorry not on msn.  I'm not saying that there is no cpu power involved.  Sure there is.  When a search is being performed 100% cpu is being used, but in these search tests that lasts only a few seconds.  See attachment showing cpu activity while searching server for mp3 files.

I logged in the server as guest when I searched for mp3 files.  Cleared cache & restarted browser.   I searched for * to give you an idea of the total folders & files searched through and the results are:  97 folders 3239 files - Total: 2.66 GB for the guest access level on my server right now.  This search took 16 seconds.  This answers foggy's total folders/files question. (attachment below).

I didn't mention this, but these tests were done locally.

[TSG]

I have a theory that its not the recursive search, but the html page generation done by HFS once it has the results, this has been seen with folders that contain a lot of files. I know for a fact that one of my wallpaper directories with about 200 files takes a lifetime to load if it hasn't been opened for a while.

I think there is 2 problems occurring at once, 1. the recursive search using 100% cpu, 2. hfs trying to generate a html page to send. The results indicated from your tests prove that having more characters, it takes HFS longer to generate the page.

[maverick]

I have a theory that its not the recursive search, but the html page generation done by HFS once it has the results, this has been seen with folders that contain a lot of files. I know for a fact that one of my wallpaper directories with about 200 files takes a lifetime to load if it hasn't been opened for a while.

We differ here.  I have a wallpaper folder with almost 1600 files and it loads in a few seconds.  (all jpg).  Actually they were included in my test above when I did a "*" search.

I think there is 2 problems occurring at once, 1. the recursive search using 100% cpu, 2. hfs trying to generate a html page to send. The results indicated from your tests prove that having more characters, it takes HFS longer to generate the page.

I agree that the more there is to display, the longer it will take to load the page which I would say is normal behavior.  But in my tests that only accounts for a few seconds more not the big time lag mentioned in other parts of this thread.

I don't know what else to tell you guys.  You have a bottleneck somewhere.

[TSG]

A bottleneck? hmm.

I just searched my entire HFS vfs for ".jpg" and HFS sat at 100% for... a lifetime...  both tog and terayon do this. The HFS default on the other hand returned results within a few seconds... this is why i think it is the html page generation. Here is some examples of the different code sizes for a table row.

Default template:

Code:

[file]
<tr><td>%new% %protected% <a href="%item-url%"><img src="/~img_file" /> %item-name%</a>%comment%<td align=right>%item-size%<td align=right>%item-modified%<td align=right>%item-dl-count%

ToG template:

Code:

[file]
<tr onmouseover="onTR(this)" onmouseout="offTR(this)">
<td class=fcl>%protected%<a href="%item-url%"><img align=top src="/~img_file" alt=""> %item-name%</a>%new%%comment%</td>
<td class=icl><div class=ht><b>Hits:</b> %item-dl-count%</div><b>Size:</b> %item-size%<div><b>Age:</b> %item-modified%</div></td>
<td class=qcl style="text-align:center"><a href="%item-url%" class=mp3>Download | <img src="/~img33" alt="DL" align=top></a></td>
</tr>

Terayon template:

Code:

[file]
</tr>
<tr>
<td class="background" height="64"><a href="%item-url%" class="messageimagelink"><div class="noIcon"><img src="/~img_file" class="image" alt="" border="0" /><span class="extension"><script type="text/javascript">writeExt("%item-ext%");</script></span></div></a></td>
<td width="100%"><div class="forumlink fileLink"><a href="%item-url%">%item-name%</a></div><span class="genmed" style="width:85%;">%comment%</span><div class="minHeight"></div>
<div class="gensmall fileStats"><span style="width:125px;float:left;"><b>File Size:</b> %item-size%</span><span style="width:65px;float:left;"><b>Hits:</b> %item-dl-count%</span><span style="width:200px;float:left;"><b>Date:</b> %item-modified%</span></div></td>
<td nowrap="nowrap" class="qrow"><div class="quicklinks"><div class=q1><a href="%item-url%" title="Download this file"><div class="dow-a">&nbsp;</div></a></div><div class=q2><div class="emp">&nbsp;</div></div></div></div></td>

Live 2:

Code:

[file]
<script language="javascript">var file_name = "%item-name%";var folderpath= "%item-folder%";var file_url= "%host%"+folderpath+file_name;filename = remplace("html",file_name);fileurl= remplace("url",file_url);fileurlstream = remplace("stream",file_url)</script>
<script language="javascript">
{document.write('<table id ="table_normal" style="width: 99%; border-style: solid;  text-align: left; height: 22px;" cellspacing="0" cellpadding="0" onmouseover="this.id=\'table_over\';" onmouseout="this.id=\'table_normal\';"  onmousedown="this.id=\'table_down\';"  onmouseup="this.id=\'table_over\';" onclick="window.location=\'//'+fileurl+'\'" ><tr>');filetypes(filename,fileurl); document.write('</a></td><td style="height: 58px"><a href="//'+fileurl+'" id="item_headertext">'+filename+' ');}
</script>%new% <br/><script language="javascript">
var showhide = readCookie('showhits');if (showhide == 2){showtime = "none";showhits= "none";}else if (showhide == 1){showtime = "visible";showhits= "visible";}else {showtime = "visible";showhits= "none";}document.write('<span id="Opb" name="Opb" style="display:'+showtime+'"><font id="item_smalltext">%item-modified%<br id="item_smalltext" /></span>%item-size%<span name ="Op" id="Op" valign="bottom" style="display:'+showhits+';  font-style: oblique; padding-left: 7px">%item-dl-count% '+txt_visits+' </span></font></a><br /><span id="Op" name="Op" style="display:'+showhits+'">');
</script><span id="item_smalltext">%comment%</span></span></td></tr></table>

The way i see it is, the larger a table row is, the longer it takes for hfs to cache up a html file to send the browser. This is a serious problem, cause in the future more complex templates are to come, I feel the ToG compared to other sites i have built is extremely lightweight considering the amount of features it has. And for a program so dynamic in nature with its possible uses, i feel that this process needs to be worked on. It is also apparent when opening a large directory of files, not only when searching, thats just triggered a noticeable effect.

[maverick]

So it is a template issue.  I may be wrong, but the no thumb image and your quicklink section might be slowing up things as a new connection might be needed for each instance. 

[rejetto]

the same gif file in an html is downloaded only once, even if used 100 times

[parade]

It must be your computer hardware. My computer runs 2 sata drives with the processor mentioned above and most searches work fine. It is the only way to search and searching will use resources, it is not a simple task to iterate through lines of text. Rejetto has tried his best to minimise the effects. And yes anything below 2ghz is so slow (to my knowledge that processor is 1.6ghz and about 6 years old).

I knew that my computer is not so slow.
After 3 weeks without any searching, I tried it again today.
All results come within some seconds. When I tried it last time I got no result at all, maybe because I wasn't so patient to wait more than 10 Minutes.

So what has changed?
If I remember right it's only the HFS-Build. Today I switched to the new Beta Build 160.