Problem Updating Large Groups?

Bugs..those nasty little critters that crawl and hide in your code at night..

Problem Updating Large Groups?

Postby plz7788 » Thu Jun 16, 2016 2:20 am

Hi guys, seems like not much has been happening here for a long time. Here is a problem I have been having.

The updating of large groups, groups with more than say... 4 million messages? Headers get downloaded up to a certain point, then the program goes into 'crunching'(?), and usually, after a while, it moves on and resumes downloading headers. But when it gets to large percentages (say, 40%, but much smaller if the group is very large), it will get stuck 'crunching' forever and never move on. When it goes into 'crunching' mode, it can draw a lot of CPU usage, so some more intense programs that I use while Newsflash is running can cause either the program, or Newsflash to crash/lag.

The solution then is to never run demanding programs while Newsflash is running, and always have Task manager ready to kill Newsflash if it starts 'crunching' forever. Also, when you kill Newsflash with Task manager, it sometimes remembers what it did before it died, and sometimes doesn't, for instance, it may have downloaded 50mb of headers before being killed, but it gets stuck - > Task manager kill -> restart Newsflash -> no headers downloaded. But sometimes, 50mb will be there, and sometimes, 40mb may be there...

This also means that, with my slow internet connection, getting all headers for groups larger than 6 million messages is nigh impossible, unless I devote myself to my computer all day, which is impractical.

Thanks for reading. Looking forward to any suggestions or fixes.
plz7788
 
Posts: 11
Joined: Mon Mar 07, 2016 10:44 pm

Re: Problem Updating Large Groups?

Postby ensisoft » Thu Jun 16, 2016 6:57 am

Hmm, this sounds obscure. I don't have any answers off the top of my head but I'll look into this.

Basically the "crunching" means that the application has received data over the network faster than it has been able to process and has temporarily stored that data in your computer's system memory while it's going through it and processing it. However 4m messages is not really that much and 40% of that is less than 2m. This shouldn't really be a problem at all.

Few questions.

- windows or linux? (which OS and what version)
- which version of newsflash?
- are you using compressed headers?
- what kind of network bandwidth are you having?
- what is your service provider?
- any specific group or is it just "any group with lots of data" ?
- are you on a SSD disk?
- can you provide specs for your hardware?

Thanks,

- ensi
ensisoft
Site Admin
 
Posts: 159
Joined: Thu Aug 19, 2010 11:02 am

Re: Problem Updating Large Groups?

Postby plz7788 » Wed Jun 22, 2016 1:13 pm

ensisoft wrote:Few questions.

- windows or linux? (which OS and what version)
- which version of newsflash?
- are you using compressed headers?
- what kind of network bandwidth are you having?
- what is your service provider?
- any specific group or is it just "any group with lots of data" ?
- are you on a SSD disk?
- can you provide specs for your hardware?

Thanks,

- ensi


Thanks.

- Windows (I'm on 10 now)
- version 4.1.1 is what the 'About' tells me
- ermmm... I don't think so? Terribly sorry, not too technologically proficient. As far as I can tell, I am using 'normal' stuff, I have not chosen to do anything special.
- currently my download speed maxes out at ~200kb/s depending on the day. When it is fast I usually see Newsflash go up to 180kb/s.
- Astraweb
- as far as I can tell, its "any group"
- physical spinning disc type HD
- i7, 8gb ram, 1tb HD (far from storage limit), GPU seems irrelevant...
plz7788
 
Posts: 11
Joined: Mon Mar 07, 2016 10:44 pm

Re: Problem Updating Large Groups?

Postby ensisoft » Sat Jun 25, 2016 8:25 am

Seems weird. I just tried this also on Astraweb. a.b.formula1 has about 8.3m headers and it completed fine. Comes out at about 84 header blocks and 50.2k items to be displayed.

The crunching basically means that the application has buffered, i.e. received more data over the network than it has been able to process, so it's taking a short break from downloading new data and just processes what It already has. So basically what you're saying is that sometimes when it goes into "crunching" it won't be able to continue.

Could you take another stab at this? You can also try turning on the compressed headers (this will quicken the download time for the headers) by opening your account properties (Edit Account) on the main tab, then go into "Advanced" tab and check the "Use compressed headers" box.
ensisoft
Site Admin
 
Posts: 159
Joined: Thu Aug 19, 2010 11:02 am

Re: Problem Updating Large Groups?

Postby simexi » Fri Jul 29, 2016 11:42 am

I've same kind of problem. While updating groups the app randomly freezes.. but continues updating. Tried a.b.formula1 and same thing... I have Eweka, win10, latest app version, 16gb, i5-4670K, 100Mbit cable...
simexi
 
Posts: 1
Joined: Fri Jul 29, 2016 10:47 am

Re: Problem Updating Large Groups?

Postby ensisoft » Sat Jul 30, 2016 7:04 am

Are you viewing the contents of the group while it is updating? If so how many of the header blocks have you loaded and how many items are there displayed in the group?

Thanks
ensisoft
Site Admin
 
Posts: 159
Joined: Thu Aug 19, 2010 11:02 am


Return to Bugs

Who is online

Users browsing this forum: No registered users and 1 guest

cron