Intermittently slow forum

Anything about use of this forum : NOT about cycling
User avatar
Paulatic
Posts: 4014
Joined: 2 Feb 2014, 1:03pm
Location: 24 Hours from Lands End

Re: Intermittently slow forum

Postby Paulatic » 30 Sep 2018, 5:33pm

My last post was taking an age to look posted. While I was waiting I opened the forum in another tab. The post was there and then seconds later looked posted on the original tab.
Whatever I am, wherever I am, this is me. This is my life

https://stcleve.wordpress.com/category/lejog/

User avatar
gaz
Posts: 13639
Joined: 9 Mar 2007, 12:09pm
Location: Kent, car park of England

Re: Intermittently slow forum

Postby gaz » 30 Sep 2018, 9:45pm

Latest contribution.

Untitled.png
Wanted ad x 6

Reported to the mods. Tidied up since, possibly by the OP.

User avatar
mjr
Posts: 13263
Joined: 20 Jun 2011, 7:06pm
Location: Norfolk or Somerset, mostly
Contact:

Re: Intermittently slow forum

Postby mjr » 1 Oct 2018, 9:02am

The random delays sound like backing off for random time after some collision for an exclusive resource so I think I'd be looking at the database server or less likely any locks in the code (unlikely in phpbb base code so maybe an extension - more common in email servers as they get busy, which presumably led to the notification-related ones that this probably isn't), checking the git or apt or similar logs for changes around the time this started... but I expect admin's trying this already.
MJR, mostly pedalling 3-speed roadsters. KL+West Norfolk BUG incl social easy rides http://www.klwnbug.co.uk
All the above is CC-By-SA and no other implied copyright license to Cycle magazine.

thirdcrank
Posts: 28648
Joined: 9 Jan 2007, 2:44pm

Re: Intermittently slow forum

Postby thirdcrank » 1 Oct 2018, 12:34pm

Posting as a total information technology numpty, I think this is being caused by thread resurrection. It must strain the system if a long-running thread from years ago is suddenly the focus of renewed attention. It's a bit like going into the back of a very full garage or shed and deciding to pull out an old mangle or a work bench. It cannot be achieved without collateral damage to the junk that's in the way, the back of the person doing the pulling or both.

User avatar
mjr
Posts: 13263
Joined: 20 Jun 2011, 7:06pm
Location: Norfolk or Somerset, mostly
Contact:

Re: Intermittently slow forum

Postby mjr » 1 Oct 2018, 12:41pm

thirdcrank wrote:Posting as a total information technology numpty, I think this is being caused by thread resurrection. It must strain the system if a long-running thread from years ago is suddenly the focus of renewed attention. It's a bit like going into the back of a very full garage or shed and deciding to pull out an old mangle or a work bench. It cannot be achieved without collateral damage to the junk that's in the way, the back of the person doing the pulling or both.

Sorry to disagree but phpbb didn't archive old topics the last I looked, so actually posting new threads would be slightly more resource-consuming - like shoving ever more stuff into the front of the shed. So the database server software would be choosing what to keep in memory and what to put to the back, which would depend on usage.

In other words, what gets shoved to the back of the shed is what's not used, regardless of how old it is. What really matters would be the number of active threads at any one time - meaning stuff is always getting pulled out the back of the shed. I don't feel that the number of active threads has actually increased all that much.
MJR, mostly pedalling 3-speed roadsters. KL+West Norfolk BUG incl social easy rides http://www.klwnbug.co.uk
All the above is CC-By-SA and no other implied copyright license to Cycle magazine.

Psamathe
Posts: 9854
Joined: 10 Jan 2014, 8:56pm

Re: Intermittently slow forum

Postby Psamathe » 1 Oct 2018, 12:46pm

Happens on sending PMs as well.

Ian

User avatar
Mick F
Spambuster
Posts: 45785
Joined: 7 Jan 2007, 11:24am
Location: Tamar Valley, Cornwall

Re: Intermittently slow forum

Postby Mick F » 1 Oct 2018, 2:50pm

I had issues earlier today and I think now, that it's not the actual posting, but the tell-back to you that it's been posted and YOUR page being updated.

I wonder if the"two computer method" may show up something?
Post on one and lurking on the other whilst continually updating the page. I wonder if it would turn up on the lurking computer before it turns up on the posting computer?

Going to try this now.
Mick F. Cornwall

User avatar
Mick F
Spambuster
Posts: 45785
Joined: 7 Jan 2007, 11:24am
Location: Tamar Valley, Cornwall

Re: Intermittently slow forum

Postby Mick F » 1 Oct 2018, 2:51pm

Both happened instantaneously, so nothing proven. :oops:
Mick F. Cornwall

thirdcrank
Posts: 28648
Joined: 9 Jan 2007, 2:44pm

Re: Intermittently slow forum

Postby thirdcrank » 1 Oct 2018, 6:15pm

... I don't feel that the number of active threads has actually increased all that much. ...


My impression is that recently forum activity has fallen (even allowing for occasional duplicate posts when somebody presses submit more than once.)

Once upon a time, clicking "unread posts" was only "no results found" during the small hours. Now it happens in mid-afternoon.

User avatar
gaz
Posts: 13639
Joined: 9 Mar 2007, 12:09pm
Location: Kent, car park of England

Re: Intermittently slow forum

Postby gaz » 2 Oct 2018, 7:28pm

Mick F wrote:Post on one and lurking on the other whilst continually updating the page. I wonder if it would turn up on the lurking computer before it turns up on the posting computer?

Posted, then opened up a new tab to view the forum. I didn't do this quickly, it was an afterthought, say 5 seconds.

Post showing in new tab. About a further 20 seconds for the original tab to return that the post had been completed.

User avatar
Mick F
Spambuster
Posts: 45785
Joined: 7 Jan 2007, 11:24am
Location: Tamar Valley, Cornwall

Re: Intermittently slow forum

Postby Mick F » 3 Oct 2018, 8:24am

Thanks.
That's what I thought may be happening.

I have no clear idea how all this works these days with the internet stuff, but it must be some sort of handshake sequence going on. Post, and the forum accepts it and displays it, but you don't see it until the forum sends it back to your computer to update your page.

Back in my computer days - 1980s main frame stuff - the peripherals would request info. They sent a request to the CPU and they would talk to each other and send various handshakes to each other as the info was passed across. It was an involved and complicated arrangement, but it was all over in milliseconds.

I have all my notes and info up in the loft, but it'll stay there! :wink:
Mick F. Cornwall

User avatar
admin
Site Admin
Posts: 1179
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Intermittently slow forum

Postby admin » 3 Oct 2018, 9:01am

The trouble is that there are a lot of asynchronous and synchronous tasks going on, and there are a lot of clients sending requests to the Forum server all the time.

Most likely candidates are:

  • Database table locking (unable to write to the table until the last write has completed)
  • Email sending (might hold the original connection for longer having finished updating the database)
  • Something I haven't though of yet

My best guess, given the symptoms, is email sending. But the change I made that I think should make that asynchronous so it doesn't hold up the web connection didn't fix the problem.

Psamathe
Posts: 9854
Joined: 10 Jan 2014, 8:56pm

Re: Intermittently slow forum

Postby Psamathe » 3 Oct 2018, 9:59am

admin wrote:The trouble is that there are a lot of asynchronous and synchronous tasks going on, and there are a lot of clients sending requests to the Forum server all the time.

Most likely candidates are:

  • Database table locking (unable to write to the table until the last write has completed)
  • Email sending (might hold the original connection for longer having finished updating the database)
  • Something I haven't though of yet

My best guess, given the symptoms, is email sending. But the change I made that I think should make that asynchronous so it doesn't hold up the web connection didn't fix the problem.

My experience is that the delays can happen even when editing (and submitting) an already posted post. And I thought no e-mails are sent when editing an existing post.

Given the issues the poor performance is causing why not announce that tomorrow there will be no e-mail notifications sent for anything and disable all e-mails and see if the problems persist. I'm sure that if people are notified losing the e-mails for a day would not be the end of the world. Better if you could somehow queue them but o idea how that might be done and just disabling them would probably be a better test?

Ian

User avatar
admin
Site Admin
Posts: 1179
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Intermittently slow forum

Postby admin » 3 Oct 2018, 10:32am

I have other ways to work out where the slowness is coming from. I don't really want to break functionality at this stage.

User avatar
admin
Site Admin
Posts: 1179
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Intermittently slow forum

Postby admin » 3 Oct 2018, 11:03am

Well, it doesn't seem to be anything to do with the phpBB code, which manages to complete all it wants to do in a few hundred milliseconds.

So it's something lower-level on the server.