URL Encoding and Decoding - Javascript vs PHP

[iwato]

BACKGROUND:  Working with Matomo has compelled me to dig deeper into the way that information gets passed over the internet.  Up until now I have assumed that I can pass just about anything in a query string.  In fact, this is exactly what I have been doing.   What I am beginning to realize is that the PHP $_POST and $_GET variables have been doing work of which I was not aware.  I have come to this conclusion with my new encounter of the encodeURI(), encodeURIComponent(), decodeURI(), and decodeURIComponent() functions.  Indeed, I have read that no information can be passed in an HTTPRequest that is not in ASCII format.  Is this true?

QUESTION:  Is it true that when working with  PHP the values of PHP HTTPRequests are automatically URL encoded and URL decoded, and that when working with Javascript the same values must be manually encoded and decoded?

Roddy

 

 

 

 

Edited April 25, 2018 by iwato

[Ingolme]

URLs can only contain ASCII characters and even among those, query string components need to have =, ? and & escaped. 

PHP superglobals have their values decoded already, but if you were to access the raw HTTP data you would need to decode it manually. If you are using PHP to generate URLs or send HTTP requests then you have to manually encode the data. 

[justsomeguy]

Also, for what it's worth, URLs do have a maximum size limit, and it's not the same across browsers.  So, be careful about how much you decide to cram into a querystring.  If you can use a post request instead, do so.

[iwato]

On 4/25/2018 at 11:16 AM, Ingolme said:

URLs can only contain ASCII characters and even among those, query string components need to have =, ? and & escaped. 

If this were true, then what would allow me to send the following link in an email and have it returned by the user in tact and process ready.

https://www.grammarcaptive.com/_utilities/php/email_verify.php?name=Lindsey%20Graham&email=kiusau@me.com&hash=$2y$10$yRg4Di9sKMYnAwq24pMya.4P7eTZctllMVpmctVGASc/siuvuUP5G&language=en&letter_no=1

Roddy

p.s. Please excuse the recipient's name, but one does not always have control over who subscribes, and it is rare that one can ever truly know the subscriber's true motivation.  Sometimes not even the subscriber knows.

 

[iwato]

On 4/25/2018 at 12:15 PM, justsomeguy said:

Also, for what it's worth, URLs do have a maximum size limit, and it's not the same across browsers.  So, be careful about how much you decide to cram into a querystring.  If you can use a post request instead, do so.

Thank you for the head's up!

Roddy

[Ingolme]

Encode the components. In PHP you can use rawurlencode(), in Javascript you would use encodeURIComponent.

Here's how it's done. A Javascript example:

var url = "http://www.example.com?a=" + encodeURIComponent(something) + "&b=" + encodeURIComponent(somethingelse);

A PHP example:

$url = "http://www.example.com?a=" . rawurlencode($something) . "&b=" . rawurlencode($somethingelse);

 

[iwato]

Yes, this is what  I thought as well.  So, why did you write

Quote

URLs can only contain ASCII characters and even among those, query string components need to have =, ? and & escaped. 

The following portion of the above appears to contradict the examples just provided.

Quote

and even among those, query string components need to have =, ? and & escaped. 

Roddy

[Ingolme]

The rawurlencode() and encodeURIComponent() functions transform any non-ASCII values into ASCII, that's exactly what they are for.

I said query string components, which refers to these sections: ?a=component1&b=component2 If you do not escape =, ? and & in those values then you will not get the values you expect in the server. The following example is a query string that will not give the desired value on the server: ?candy=M&Ms. If you do not escape the &, then the server will see a key "candy" with value "M" and another key "Ms" with no value.

[iwato]

Got it!  So, I have been lucky in my own query string composition and did not know it.

Roddy

[justsomeguy]

I suppose it would be more correct to say that any metacharacter that is part of your data needs to be escaped.  So if the data you are passing contains metacharacters like "#", "%", "&", "?", etc, as part of the data then those characters need to be escaped. They have special meanings when used in a URL, just like ":", "/", "@", etc.