schalli/tessera-ctl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 14 Wiki Activity

fix(favorites): use realistic browser User-Agent for icon byte-fetch

Browse Source 
Reproducibly confirmed (3/3 vs 3/3 direct comparison inside the API
container) that chatgpt.com's Cloudflare WAF returns 403 for the
"tessera/1.0" User-Agent regardless of Accept header, and 200 for a
real Chrome UA string. Parameterized fetchWithRedirectGuard's
User-Agent (defaulting to the existing "tessera/1.0") and override it
only for fetchIconBytes -- the HTML-discovery path (discoverFavoriteIconUrl)
keeps its original User-Agent unchanged, per the no-regression constraint
on that flow.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Schalli
2026-07-07 15:41:32 +02:00
parent 30d6e0a5df
commit f06a2ff397
1 changed files with 10 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/api/src/favorites/icon-discovery.service.ts
+10 -6
View File
@@ -227,7 +227,7 @@ function extractIconFromHtml(html: string, baseUrl: string): string | null {
*/ */
async function fetchWithRedirectGuard( async function fetchWithRedirectGuard(
pageUrl: URL, pageUrl: URL,
options: { accept: string; timeoutMs: number }, options: { accept: string; timeoutMs: number; userAgent?: string },
): Promise<{ response: Response; finalUrl: URL } | null> { ): Promise<{ response: Response; finalUrl: URL } | null> {
let currentUrl = pageUrl; let currentUrl = pageUrl;
@@ -245,7 +245,7 @@ async function fetchWithRedirectGuard(
signal: controller.signal, signal: controller.signal,
headers: { headers: {
Accept: options.accept, Accept: options.accept,
'User-Agent': 'tessera/1.0', 'User-Agent': options.userAgent ?? 'tessera/1.0',
}, },
}); });
@@ -330,11 +330,15 @@ export class IconDiscoveryService {
const url = new URL(iconUrl); const url = new URL(iconUrl);
const result = await fetchWithRedirectGuard(url, { const result = await fetchWithRedirectGuard(url, {
// A bare "image/*" Accept header (paired with our non-browser // A browser-realistic Accept header and User-Agent avoid tripping
// User-Agent) trips bot-mitigation WAFs on some sites (observed: // bot-mitigation WAFs that block non-browser clients (observed
// chatgpt.com/favicon.ico returns 403 with this combo) — a realistic // reproducibly: chatgpt.com/favicon.ico returns 403 for our default
// browser-style image Accept list avoids that false positive. // "tessera/1.0" UA and 200 for a real Chrome UA string, confirmed by
// repeated direct comparison). The HTML-discovery path is unaffected —
// this User-Agent override only applies to this byte-fetch call.
accept: 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8', accept: 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
userAgent:
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
timeoutMs: ICON_FETCH_TIMEOUT_MS, timeoutMs: ICON_FETCH_TIMEOUT_MS,
}); });