Selective Proxying to debug live website issues.

I was recently working on debugging a live website issue; it was only reproducible on the live website, and it was challenging to debug the issue with the live site. Based on my research, I identified selective proxying — modifying the response for the selected URLs and passing the rest of the traffic to the live URL- helping debug live website issues quickly. I want to share some possible tools and configurations that can be used to debug live website issues through selective proxying.

The actual issue was the JavaScript embedded into the HTML page was not working as expected; to debug the problems, the script in the HTML should be modified. Finally, I used the Fiddler Autoresponder to serve only the specific HTML from the Local File and the remaining URLs through live websites. This helps me modify the embed script, identify the root of the problem, and address it.

Fiddler Autoresponder:

The Auto Responder feature in Fiddler Everywhere allows you to create “Rules” triggered when a particular request is issued. Fiddler’s Autoresponder will enable you to return files from your local disk instead of transmitting the request to the server.

The Fiddler can be downloaded from Download Fiddler Web Debugging Tool for Free by Telerik.

Select Autoresponder from Fiddler UI and click Add Rule.

Add the string to match — the URL that should be responded to with local file content.

Select Find File and locate the local File — in my case, I will replace the response for one the HTML — copy the actual HTML response through the view page source in the live URL, store it in the local File, and modify the content as required.

Save and Enable the Rule

Enable the “Decrypt HTTPS Traffic” option if you are debugging HTTPs websites.

Tools → Options → HTTPS

I am selecting No for the root certificate; if the root certificate is not accepted, you will receive an untrusted SSL certificate error while accessing the website — the root certificate can be trusted later if required.

I am enabling the Decrypt only for the browsers.

Click Ok and access the website(page); the specific page will be rendered from a local file — I have enabled a quick alert on the local File.

Now the specific HTML file is loaded locally; all remaining requests are responded to from the live server— including the relative links inside the HTML page loaded from locally.

The SSL certificate error may block some of the externally linked resources from loading; if you want, you can trust the Fiddler root certificate (keep security in mind)

Go to Tools → Option → HTTPS, Click on Actions → Trust Root Certificate.

Follow the screens and Trust the certificate; now, the SSL error will be removed.

You can remove the root certificate if required through Tools →Options → HTTPS →Actions →Reset All Certificates.

The Fiddler is enabled system-wide to capture all the traffic; you can modify the configurations to capture only the specific traffic. Also, you can pause the capturing by unselecting the File →Capture Traffic configuration.

Requestly:

Another option is using the Requestly Desktop app to proxy the response; the Requestly Desktop app can be downloaded from Desktop App to Intercept & Modify HTTP Requests | Requestly

First, open the browser profile (recommend new profile) through the connected app.

Now enable an HTTP Rule.

Select Modify Response Rule

Add the URL for the response that should be replaced — refer to the previous option for replacing HTML with local content.

Place the updated content (in my case, HTML) in the response body.

Provide a Rule name and Save the Rule (Ctrl+S)

Now test the page through the browser session opened by Requestly. For the matching page, the response body is replaced by content specified in the Rule, and the remaining links are directed to the live server.

The Requestly is simple to configure and restricted to a specific browser session, but the Fiddler will give more functionalities like traffic monitoring, SSL support, etc. You can even use other proxies like Charles proxy etc.; consider the security before using any tools for proxying the requests- use the proxies only when you need some debugging.