Syntax error throwing formatting out

Anything about use of this forum : NOT about cycling
Psamathe
Posts: 12247
Joined: 10 Jan 2014, 8:56pm

Re: Syntax error throwing formatting out

Postby Psamathe » 18 Sep 2017, 3:17pm

Forum often "running like a dog" today so I thought I'd analyse the loading and piwik is taking over a third of the page load time! (0.8 secs out of a 2.1 sec load time) and 20% of the download
Screen Shot 2017-09-18 at 15.10.03.png

Whilst piwik.js seems to load in parallel with other stuff, the https://piwik.fonant.com/piwik.php?action_name=Cycling%20UK%20Forum%20-%20Login&idsite=34143&rec=1&r=416776&h=16&m=2&s=46&url=https%3A%2F%2Fforum.cyclinguk.org%2Fsearch.php%3Fsearch_id%3Dnewposts&_id=f5bedb1353bca147&_idts=1505743367&_idvc=1&_idn=0&_refts=0&_viewts=1505743367&send_image=1&pdf=1&qt=0&realp=0&wma=0&dir=0&fla=1&java=1&gears=0&ag=0&cookie=1&res=1280x1024&gt_ms=67&pv_id=84bu3q seems to take place pretty well after everything else has downloaded and it's that that is taking the extra 0.8 secs.

Although that does not explain the delays I'm seeing as I've blocked piwik resources/URLs. Maybe for me it's just having a "bad day".

Ian

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 18 Sep 2017, 4:05pm

Thanks for letting me know.

I've looked at the server monitoring graphs, and at Piwik's client-side monitoring, and there aren't any problems with the server. Forum pages are being generated and downloaded by people's browsers in a time of between 0.2 (for the Home page) and 0.3 seconds (for search results pages).

So I think it must be something more local to your machine. I recommend Ping Plotter free (https://www.pingplotter.com/products/free.html) as a tool to see if there are network problems between my browser and a web site.

Psamathe
Posts: 12247
Joined: 10 Jan 2014, 8:56pm

Re: Syntax error throwing formatting out

Postby Psamathe » 18 Sep 2017, 4:11pm

admin wrote:Thanks for letting me know.

I've looked at the server monitoring graphs, and at Piwik's client-side monitoring, and there aren't any problems with the server. Forum pages are being generated and downloaded by people's browsers in a time of between 0.2 (for the Home page) and 0.3 seconds (for search results pages).

So I think it must be something more local to your machine. I recommend Ping Plotter free (https://www.pingplotter.com/products/free.html) as a tool to see if there are network problems between my browser and a web site.

The delays come and go so I'll only investigate further if they persist e.g. tomorrow (plus we've had a BT van in the area last week and they said they'd be here this week as well!!!

It was more the resource used by piwik that surprised me (as I said, not related to my own observations as it's blocked) but a significant %age of data/load time.

Ian

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 18 Sep 2017, 4:19pm

If your connection is suffering from a lot of packet loss, data can take random lengths of time to reach you. If the packets get through first time it's nice and snappy, the more packets get lost for a particular connection the longer it takes as the lost packets have to be re-transmitted. Which packets actually get lost is random, hence the random fluctuation in download times.

Random website delays that are noticeable are, I think, more likely to be network issues leading to packet loss than server problems. Servers tend to be either happy or overloaded, and server load tends to change relatively slowly over time.

User avatar
mjr
Posts: 16717
Joined: 20 Jun 2011, 7:06pm
Location: Norfolk or Somerset, mostly
Contact:

Re: Syntax error throwing formatting out

Postby mjr » 18 Sep 2017, 4:31pm

admin wrote:I've looked at the server monitoring graphs, and at Piwik's client-side monitoring, and there aren't any problems with the server. Forum pages are being generated and downloaded by people's browsers in a time of between 0.2 (for the Home page) and 0.3 seconds (for search results pages).

So I think it must be something more local to your machine. I recommend Ping Plotter free (https://www.pingplotter.com/products/free.html) as a tool to see if there are network problems between my browser and a web site.

I don't think it's local to Psamathe. As well as being on the other side of Norfolk and connected via Cambridgeshire rather than Norwich, I think I use a different ISP with a different backend network. My connection doesn't suffer much packet loss most of the time and I would spot that on my reflashed router.

If I am foolish enough to permit piwik.fonant.com to execute javascripts, page loads slow down with about 0.6s attributed by Firefox's network monitoring to pwik.js (which is cacheable) and 0.3s for piwik.php (which isn't). Average page load times for me slow from about 1.6s to 2.6s. May I suggest that the times reported to piwik are for forum page loads and don't include the time taken to communicate with piwik? Unless it reports the previous page's piwik time, I don't think they can because the piwik time hasn't yet elapsed at the time that the browser sends its message to piwik.
MJR, mostly pedalling 3-speed roadsters. KL+West Norfolk BUG incl social easy rides http://www.klwnbug.co.uk
All the above is CC-By-SA and no other implied copyright license to Cycle magazine.

Psamathe
Posts: 12247
Joined: 10 Jan 2014, 8:56pm

Re: Syntax error throwing formatting out

Postby Psamathe » 18 Sep 2017, 4:56pm

mjr wrote:
admin wrote:I've looked at the server monitoring graphs, and at Piwik's client-side monitoring, and there aren't any problems with the server. Forum pages are being generated and downloaded by people's browsers in a time of between 0.2 (for the Home page) and 0.3 seconds (for search results pages).

So I think it must be something more local to your machine. I recommend Ping Plotter free (https://www.pingplotter.com/products/free.html) as a tool to see if there are network problems between my browser and a web site.

I don't think it's local to Psamathe. As well as being on the other side of Norfolk and connected via Cambridgeshire rather than Norwich, I think I use a different ISP with a different backend network. My connection doesn't suffer much packet loss most of the time and I would spot that on my reflashed router.

If I am foolish enough to permit piwik.fonant.com to execute javascripts, page loads slow down with about 0.6s attributed by Firefox's network monitoring to pwik.js (which is cacheable) and 0.3s for piwik.php (which isn't). Average page load times for me slow from about 1.6s to 2.6s. May I suggest that the times reported to piwik are for forum page loads and don't include the time taken to communicate with piwik? Unless it reports the previous page's piwik time, I don't think they can because the piwik time hasn't yet elapsed at the time that the browser sends its message to piwik.

The bulk of the piwik delay seems to be a wait on the https://piwik.fonant.com/piwik.php?action_name=... stage at the end of the page load. And that seems variable e.g.
Screen Shot 2017-09-18 at 16.42.54 copy.jpg
(chopped of the earlier loads)
Yellow is "wait" and green is "receive". In that test; the page load time was 4.38 secs (not on my ISP connection). Repeated again and load time 4.02 secs (again, not through my ISP, not through BT lines ...)

Ian

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 18 Sep 2017, 6:15pm

That's odd. The server is certainly running fine and normally.

What do you get with Ping Plotter, pinging forum.cyclinguk.org?

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 18 Sep 2017, 7:30pm

The Forum is snappy from here, with or without blocking Piwik and Google Analytics.

piwik.js should take no time to load after the first time as it's cached locally. It's not large, anyway, at 62kbytes.

piwik.php is taking around 100 milliseconds to return here: 13 ms connecting, 0 ms sending the request, 99 ms waiting for the response, 9ms to receive the 43 bytes of response (a 1x1 pixel GIF image).

In any case, the Piwik code, like Google Analytics, is run asynchronously so it doesn't get in the way of the web pages loading as much as it would otherwise.

UPDATE: I've enabled compression on piwik.js, bringing its size down from 62kB to 22kB. Which will speed things up a tiny bit! :)

UPDATE: I've enabled cache-control headers, allowing a cache time of a week, for images and CSS files, which speeds things up quite a lot ! :D

Psamathe
Posts: 12247
Joined: 10 Jan 2014, 8:56pm

Re: Syntax error throwing formatting out

Postby Psamathe » 19 Sep 2017, 10:53am

admin wrote:The Forum is snappy from here, with or without blocking Piwik and Google Analytics.

piwik.js should take no time to load after the first time as it's cached locally. It's not large, anyway, at 62kbytes.

piwik.php is taking around 100 milliseconds to return here: 13 ms connecting, 0 ms sending the request, 99 ms waiting for the response, 9ms to receive the 43 bytes of response (a 1x1 pixel GIF image).

In any case, the Piwik code, like Google Analytics, is run asynchronously so it doesn't get in the way of the web pages loading as much as it would otherwise.

UPDATE: I've enabled compression on piwik.js, bringing its size down from 62kB to 22kB. Which will speed things up a tiny bit! :)

UPDATE: I've enabled cache-control headers, allowing a cache time of a week, for images and CSS files, which speeds things up quite a lot ! :D

Not important but the analysis I posted was not from my computer, not over my ISP and thus included piwik. My own system block piwik so they don't even load (hence I would not be suffering that delay).

This is my 1st post today (so I've no idea how long it will take to post on pressing "Submit") but yesterday the really really long delays were on posting. Press submit and go and make a coffee (almost literally).

I'm only posting to draw it to your attention, not "making a complaint".

EDIT: Posting (Pressing "Submit") and it went through quickly; normal speed.

Ian

Psamathe
Posts: 12247
Joined: 10 Jan 2014, 8:56pm

Re: Syntax error throwing formatting out

Postby Psamathe » 19 Sep 2017, 11:00am

admin wrote:That's odd. The server is certainly running fine and normally.

What do you get with Ping Plotter, pinging forum.cyclinguk.org?

I don't know Ping Plotter and from their web site it looks like software/services you install and purchase. There was a "free" "Fix your network" section but it has all the appearances of these dubious "Scan your computer (for malware)" scams so I wouldn't go there.

If I just do a standard Traceroute

Code: Select all

Traceroute has started…

traceroute to ctcforum.fonant.com (89.145.112.122), 64 hops max, 72 byte packets
 1  10.4.0.1 (10.4.0.1)  37.043 ms  34.052 ms  32.902 ms
 2  vlan222.core-dc2.man4.uk.m247.com (84.39.117.49)  34.039 ms  33.386 ms  34.232 ms
 3  te-9-1-0.core-dc2.man4.uk.m247.com (77.243.185.76)  34.044 ms  33.348 ms  32.768 ms
 4  te-4-5-0.bb1.man2.uk.m247.com (77.243.185.137)  39.382 ms  39.279 ms  40.792 ms
 5  xe-0-0-3-0.core1.lon2.uk.m247.com (193.27.65.165)  39.368 ms  41.289 ms  40.605 ms
 6  eth-11-1-0.pni1.lon2.uk.m247.com (77.243.176.217)  40.033 ms  40.800 ms  40.527 ms
 7  xe-0-0-3.border-1.sov.lon.uk.as29017.net (195.66.224.141)  40.893 ms  39.667 ms  39.483 ms
 8  ae3.core-1.maylands.hml.uk.as29017.net (89.145.125.38)  41.898 ms  41.216 ms  40.596 ms
 9  ae3.core-1.centro.hml.uk.as29017.net (89.145.125.78)  40.693 ms  41.313 ms  42.051 ms
10  83-223-97-54.as29017.net (83.223.97.54)  43.097 ms  40.922 ms  42.045 ms
11  olive.fonant.com (89.145.112.122)  43.233 ms  116.075 ms  41.897 ms


(But that was today and not noticed any delays today)

Ian

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 19 Sep 2017, 11:20am

Ping Plotter is just an easy way to run a traceroute, with a nice graphical output. It is free to use, although it offers more features for a fee, and it is usable by people who have no idea about computers which is why I usually recommend it.

Here's a trace from my ADSL connection to the Forum server:

forum.cyclinguk.org.png


It shows a random fluctuation in connection latency, but no packet loss (other than at the hop that is the local telephone exchange, which doesn't respond to pings). The moving-average latency is well within the green 100 millisecond range, so the connection is fine.

If you could run traceroute next time you notice the Forum being sluggish, that would be useful. Either to rule out network issues, or to diagnose them as the problem.

User avatar
mjr
Posts: 16717
Joined: 20 Jun 2011, 7:06pm
Location: Norfolk or Somerset, mostly
Contact:

Re: Syntax error throwing formatting out

Postby mjr » 19 Sep 2017, 2:51pm

admin wrote:The Forum is snappy from here, with or without blocking Piwik and Google Analytics.

piwik.js should take no time to load after the first time as it's cached locally. It's not large, anyway, at 62kbytes.

True, but it does show up in performance tests.

admin wrote:piwik.php is taking around 100 milliseconds to return here: 13 ms connecting, 0 ms sending the request, 99 ms waiting for the response, 9ms to receive the 43 bytes of response (a 1x1 pixel GIF image).

I'm getting 98ms DNS, 0ms for connecting, sending, 131ms waiting and then 0ms receiving. Looking at the DNS delay more closely, I think fonant.com's NS record has a 300-second Time-To-Live which seems short unless you're changing nameservers often? Also, the forum.cyclinguk.org CNAME looks like a 207-second Time-To-Live, which is also going to hinder DNS caching and slow down requests. I don't know what browsers currently do if cache-control headers try to suggest a longer cache time than DNS TTL - do you?

admin wrote:In any case, the Piwik code, like Google Analytics, is run asynchronously so it doesn't get in the way of the web pages loading as much as it would otherwise.

That's between you and the templates, but I don't think Firefox is displaying the page until it's loaded. I've not spotted the img returned by piwik.php being used in the page source, though!

Interestingly, it looks like file.php?avatar=.... is rather slow to return on some pages and is called multiple times. I've not yet figured out if there's a pattern to what's slow but I guess file.php may be resizing graphics so is there any simple way for it to cache what it's sending?

admin wrote:UPDATE: I've enabled compression on piwik.js, bringing its size down from 62kB to 22kB. Which will speed things up a tiny bit! :)

UPDATE: I've enabled cache-control headers, allowing a cache time of a week, for images and CSS files, which speeds things up quite a lot ! :D

They should both help but won't actually show up in some performance measurement tools.

Traceroutes and pings show no particular latency or packet loss problems, but they are slow to start - is that because of the DNS TTLs too?
MJR, mostly pedalling 3-speed roadsters. KL+West Norfolk BUG incl social easy rides http://www.klwnbug.co.uk
All the above is CC-By-SA and no other implied copyright license to Cycle magazine.

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 19 Sep 2017, 3:19pm

mjr wrote:I'm getting 98ms DNS, 0ms for connecting, sending, 131ms waiting and then 0ms receiving.


All of which are quite reasonable.

mjr wrote:Looking at the DNS delay more closely, I think fonant.com's NS record has a 300-second Time-To-Live which seems short unless you're changing nameservers often?


Nope, my domains all have 3600 seconds TTL, or an hour. So you can reference fonant.com for an hour before your DNS needs to ask the domain nameservers for an IP again. On the other hand, if I change the IP address, it will take visitors up to an hour to eventually ask again and get the new IP address.

You're probably just seeing that you last requested the fonant.com name server IP addresses 55 minutes ago, so there's only 300 seconds left before your cache will have to ask again.

To get the actual original TTL, ask one the authoritative name servers for the zone, something like:

Code: Select all

dig -t NS fonant.com                      ; get the zone's name servers
dig -t NS fonant.com @ns0.fonant.com      ; send the DNS query to one of the name servers


The response will give the full TTL.

mjr wrote:Also, the forum.cyclinguk.org CNAME looks like a 207-second Time-To-Live, which is also going to hinder DNS caching and slow down requests. I don't know what browsers currently do if cache-control headers try to suggest a longer cache time than DNS TTL - do you?


Nope, it has a 1800 TTL.

Again, I think you're looking at how much of the TTL you have left to run at your caching name server before you must ask the cyclinguk.org name servers for the answer to where forum.cyclinguk.org can be found.

mjr wrote:Interestingly, it looks like file.php?avatar=.... is rather slow to return on some pages and is called multiple times. I've not yet figured out if there's a pattern to what's slow but I guess file.php may be resizing graphics so is there any simple way for it to cache what it's sending?


You'd have to ask the phpBB3 developers, or look at the source code. I would be very surprised if they thought that resizing avatars on the fly every time one is needed was a good idea.

mjr wrote:
admin wrote:UPDATE: I've enabled compression on piwik.js, bringing its size down from 62kB to 22kB. Which will speed things up a tiny bit! :)

UPDATE: I've enabled cache-control headers, allowing a cache time of a week, for images and CSS files, which speeds things up quite a lot ! :D

They should both help but won't actually show up in some performance measurement tools.[/quote]

Depends what your performance measurement tools are measuring. If you use the developer tools in Firefox, for example, you can see that caching icons saves a large number of redundant requests to the server to see if the icons have changed (which they haven't).

mjr wrote:Traceroutes and pings show no particular latency or packet loss problems, but they are slow to start - is that because of the DNS TTLs too?


You could find out by using the server IP address instead of its name, so that DNS doesn't need to be used at all.

User avatar
admin
Site Admin
Posts: 1261
Joined: 14 Dec 2006, 8:27pm
Location: Lancing, West Sussex
Contact:

Re: Syntax error throwing formatting out

Postby admin » 19 Sep 2017, 3:29pm

Oh, if you use "dig" to look things up it'll tell you exactly how long the DNS lookup took. For example:

Code: Select all

dig forum.cyclinguk.org @ns0.dnsmadeeasy.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.62.rc1.el6_9.4 <<>> forum.cyclinguk.org @ns0.dnsmadeeasy.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22885
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;forum.cyclinguk.org.           IN      A

;; ANSWER SECTION:
forum.cyclinguk.org.    1800    IN      CNAME   ctcforum.fonant.com.

;; Query time: 12 msec
;; SERVER: 208.94.148.2#53(208.94.148.2)
;; WHEN: Tue Sep 19 15:05:49 2017
;; MSG SIZE  rcvd: 70


says that looking up forum.cyclinguk.org at ns0.dnsmadeeasy.com took a whole 12 milliseconds in total. Not a delay anyone would notice.

User avatar
mjr
Posts: 16717
Joined: 20 Jun 2011, 7:06pm
Location: Norfolk or Somerset, mostly
Contact:

Re: Syntax error throwing formatting out

Postby mjr » 19 Sep 2017, 4:36pm

admin wrote:
mjr wrote:Traceroutes and pings show no particular latency or packet loss problems, but they are slow to start - is that because of the DNS TTLs too?


You could find out by using the server IP address instead of its name, so that DNS doesn't need to be used at all.

That was interesting. Running pings and traceroutes, I've just noticed it taking different routes once it gets to as29017. It seems the same to ae3.core-2.maylands.hml.uk.as29017.net (89.145.125.34), but sometimes it then goes to 83-223-97-54.as29017.net (guess! ;-) ) and then olive.fonant.com, where other times it goes to ae3.core-2.centro.hml.uk.as29017.net (89.145.125.70) first. I wonder if one of the routes through as29017 is faster than the other.
MJR, mostly pedalling 3-speed roadsters. KL+West Norfolk BUG incl social easy rides http://www.klwnbug.co.uk
All the above is CC-By-SA and no other implied copyright license to Cycle magazine.