Understanding URL Query Strings and Encoding/Decoding

After working for a long time, I recently encountered an issue where some files couldn't be downloaded successfully from a service. The system logic involves allowing file uploads and downloads. When a customer uploads a file, a UUID is generated, and the customer's information is written into the database along with storing the file. During the download process, a JWT (JSON Web Token) containing time-limited information is generated in real-time based on the user's information and the file's UUID, to distribute a one-time download URL.

However, the JWT is carried by the frontend as a query string when calling the file download API. In the user interface, there are often download errors occurring, while directly calling the API using Swagger results in successful downloads.

After a long search, I discovered that the issue was caused by passing URL parameters containing special characters. Special characters in a URL typically carry different meanings.

Here are a few examples:

1. ?： Commonly found at the beginning of a URL or between the path and query string, it serves to separate the primary components of a URL from query parameters. In a URL, the part after '?' is typically the query string, used to pass parameters and data to the server.

For example：https://www.example.com/search?query=example

In the above URL, the 'query=example' after the '?' is the query string used to pass search parameters.

2. &： In the URL's query string, it serves to separate different query parameters, allowing the transmission of multiple parameters within the same URL.

For example：https://www.example.com/search?query=example&page=2

In this example, '&' is used to separate the query parameters 'query=example' and 'page=2'.

3. =： In the URL's query string, it serves to separate parameter names and values, linking each parameter's name with its corresponding value.

For example：https://www.example.com/search?query=example

Here, '=' separates 'query' (parameter name) from 'example' (parameter value).

If the query string carries parameters containing such special characters, browsers generally encounter issues while parsing them.

There are also other characters such as '+' and space that require careful handling when passing in a URL.

So, if you insist on passing information containing such special characters through a query string, you can address this by using URL encoding and decoding.

In the situation I encountered, it's necessary to pass a JWT through the query string. As JWTs contain numerous special characters, generating the JWT on the backend, encoding it, and then returning it to the frontend allows for proper API file downloads.

Example :

If the parameter to be carried is: =hello+world.

Without encoding, using the query string to carry parameters will lead to misinterpretation by the browser.

After encoding, it becomes: %3Dhello%2Bworld.

When returned to the frontend, there won't be any issues when calling.

As for the backend, most backend frameworks or languages' HTTP request handlers automatically decode URL parameters. However, it's still advisable to verify during usage to prevent potential errors that might require debugging later on.

It's a very basic and simple concept, so I haven't bothered to figure it out before. This lack of understanding took me half an hour to debug it, so I'll record it.😎

Web

Alvin

Software engineer, interested in financial knowledge, health concepts, psychology, independent travel, and system design.

Implementing Site Search with Fuse.js in Next.js 14

Implementing Multilingual i18n with Next.js 14

Using Next.js 14 to Render Markdown Data

Switching from Hugo to Next.js 14: Building a Blog

Next.js 14 Environment settings, Set up, Project structure

ASP.NET Core API & Blazor Implements Caching Mechanism to Improve Performance