Technical SEO: Meta tags, sitemap, robots.txt and Open Graph

In short: Search engines and social networks need clear signals: sensible titles and descriptions, a sitemap, a suitable robots.txt, canonical URLs and Open Graph data for preview images.

In more detail: Good content is of little use if it is not found, understood and shared nicely. The technical basics are manageable.

In Depth

Title and description

Every page needs a unique <title> (50 to 60 characters, topic first) and a meta description (about 120 to 160 characters, benefit-oriented). Both appear in search results. One <h1> per page, heading hierarchy, speaking URLs (/wiki/subnetting), lang attribute and alt texts.

Sitemap and robots.txt

• sitemap.xml lists indexable URLs (with lastmod); in Next.js app/sitemap.ts. Submit in Search Console.
• robots.txt controls crawlers: Disallow: /admin/, Sitemap: https://…/sitemap.xml. Caution: robots.txt is no access barrier (paths are publicly readable) and only prevents crawling, not necessarily indexing; for “not in Google” use noindex (meta or header).
• Crawl-delay is not respected by all.

Canonical, multilingual, redirects

• <link rel="canonical"> names the preferred URL for duplicates (parameters, www/without).
• hreflang links language versions (de, en).
• 301 redirect for moved pages, 404/410 for deleted ones; no chains.
• HTTPS and fast load time (Core Web Vitals) are ranking signals.

Open Graph, structured data

• Open Graph (og:title, og:description, og:image 1200×630, og:url) and Twitter cards determine the preview when sharing (WhatsApp, Discord, LinkedIn, X).
• Structured data (JSON-LD, schema.org): article, product, FAQ, breadcrumb, organisation; enable rich results. Mark up only truthful data.
• Accessibility and quality help SEO: understandable texts, internal links, mobile-friendly. Check with Search Console, Lighthouse, rich results test, opengraph.xyz.

See also: SEO basics, Lighthouse and Core Web Vitals, Rendering strategies, HTTP headers