Author Topic: Slow Rom Browser Refresh and MD5  (Read 804 times)

0 Members and 1 Guest are viewing this topic.

Offline RadeonUser

  • PJ64 Dever
  • Full Member
  • *****
  • Posts: 229
  • Karma: +1/-0
  • SuperFly Deving Guy
Slow Rom Browser Refresh and MD5
« on: September 12, 2011, 03:21:59 AM »
A few weeks ago Gent pointed out PJ64 1.6.1 was slower when refreshing the rom browser.
Initially I dismissed it and explained that release and debug were going to be slower due to the extra memory leak checking that's turned on by default and thought nothing else of it.

Today I compared PJ64 1.6.1 to 1.6 and saw that 1.6.1 is about 2 to 3 times slower depending on how many fields the rom browser is showing.
I can't explain why PJ64 1.6 is faster, the only conclusion I can reach is that Zilmar changed the code before releasing 1.6

Whatever the case may be this leaves me having to rewrite the sections of code that are causing the slowdown.
This is actually quite coincidental as I have to rewrite parts of these sections either way to support MD5 checksums.

I'd say I'll probably have this out within a day or two depending on how fancy I want to be with the code.
The first version I release will likely not support MD5 since I will want to verify that it is properly reading the values needed by the rom browser.

Offline RadeonUser

  • PJ64 Dever
  • Full Member
  • *****
  • Posts: 229
  • Karma: +1/-0
  • SuperFly Deving Guy
Re: Slow Rom Browser Refresh and MD5
« Reply #1 on: September 22, 2011, 10:23:46 PM »
Okay, this build has a much faster rom browser.
It indexes the files (rdb, rdn, rdx) temporarily to speed up the search for each entry.

There's a reason why I took so long to upload this and I will explain it now.
It looks like 1.6's rom browser, as well as 1.6.1's, suffers from a refresh problem.
If you refresh the rom browser enough it will eventually stop redrawing itself, you may notice this by either a black screen where the roms should be listed or by the rom browser never refreshing.

This past week I've been madly going over my code thinking that issue was caused by what I had added.
Today I realized this issue was also in 1.6 and was not caused by my code.
So before I add MD5 support I will be tracking down this refresh issue and fixing it.

The code I have added should work transparently, nothing special is required except for a few hundred kilobytes of memory.
So if you edit an entry and refresh the rom browser the entry should still be updated.
Please make sure that still happens.

Offline RadeonUser

  • PJ64 Dever
  • Full Member
  • *****
  • Posts: 229
  • Karma: +1/-0
  • SuperFly Deving Guy
Re: Slow Rom Browser Refresh and MD5
« Reply #2 on: September 26, 2011, 05:17:36 PM »
Fixed the GDI leak that caused the rom browser to become corrupted after approximately 10,000 roms were displayed.
The display of 10,000 roms was cumulative as the old GDI objects were never released when the roms were loaded.

Instead of simply patching the issue I overhauled the way the rom browser highlights the text.
Now only one GDI object is created each time an entry on the rom browser is selected.

The rewrite should slightly improve the rom browser refresh but it is probably less than a second.
There may be a slight performance hit when loading PJ64 but it shouldn't be much.

Offline Gent

  • PJ64 Dever
  • Sr. Member
  • *****
  • Posts: 463
  • Karma: +2/-0
  • SuperFly Deving Guy
    • Project64 Website
Re: Slow Rom Browser Refresh and MD5
« Reply #3 on: September 26, 2011, 05:25:17 PM »
real nice just refreshed 30 times one after the other! the most was 7-8 times before black screen (no rom browser) Great work RAD!

Offline RadeonUser

  • PJ64 Dever
  • Full Member
  • *****
  • Posts: 229
  • Karma: +1/-0
  • SuperFly Deving Guy
Re: Slow Rom Browser Refresh and MD5
« Reply #4 on: December 25, 2011, 03:11:59 AM »
Removed an optimization for reading the support files.

I used to read the first '[' followed by it's matching end bracket of ']'
To make things faster I simply incremented the reading routine by the amount that was read, after all if it didn't match an identifier ([3B941695-F90A5EEB-C:50] for example) then there was no point in examining that section again.

Well it turns out Gent has been adding [] into the good name.
Normally this isn't an issue but he mistyped a few of the closing brackets (or simply forgot them) so my optimization would read a large chunk, possibly up to the next entry in the support files, and not load the settings/cheats/whatever for that rom since I was incrementing past the ] I had read.

This shouldn't change performance much as I read large chunks from the file into memory.
All this does is cause a little extra cpu usage but I doubt it will be noticed.

Edit: Here's a list of the entries that are 'broken' for Gent's convenience.
[BB752DA7-30D8BEF4-C:0] has Internal Name=ór [xg
[668FA336-2C67F3AC-C:0] has Good Name=Pip's RPGs Beta x (PD) [a1
[DD95F49D-2A9B8893-C:0] has Good Name=NBC-LFC Kings of Porn Vol 01 [a1} (PD)

This edit actually gave me a better look at the entries on the rdb and it seems like internal name is completely unnecessary.
The only time it's not fetched from the rom is when PJ64 first starts and reads that information from the cache file.
Otherwise all dialogs and prompts partially load the rom (if not already loaded) and reads it from the rom directly.
« Last Edit: December 25, 2011, 03:36:10 AM by RadeonUser »

Offline Gent

  • PJ64 Dever
  • Sr. Member
  • *****
  • Posts: 463
  • Karma: +2/-0
  • SuperFly Deving Guy
    • Project64 Website
Re: Slow Rom Browser Refresh and MD5
« Reply #5 on: December 25, 2011, 09:28:14 PM »
Yea i need to go over all the Good Names and correct any wrong ones to try to match the File name etc.


As for the Broken ones (Showing up in the browser as bad althought it seems i have the right crc and good name in place leaves a little :|


I will correct the above entries with the following:



[BB752DA7-30D8BEF4-C:0]
Good Name=LaC's Universal Bootemu V1.0 (PD)


[BB752DA7-30D8BEF4-C:0]
Good Name=LaC's Universal Bootemu V1.1 (PD)




[668FA336-2C67F3AC-C:0]
Good Name=Pip's RPGs Beta x (PD) [a1]


[DD95F49D-2A9B8893-C:0]
Good Name=NBC-LFC Kings of Porn Vol 01 (PD) [a1]