Add Website URL

Overview#

Add Website URL ingests publicly accessible web pages into your knowledge base. Provide a link and the system crawls the page content so agents can reference it without manual copying.

When to Use It#

• Capturing product documentation already published on your site
• Pulling in blog articles or changelog updates as training material
• Syncing help center resources without exporting files

Key Benefits#

• Automatic capture — paste a URL and let SotaAgents retrieve the page content
• Fresh information — quickly update the knowledge base when the website changes
• Hands-off workflow — no need to download or reformat HTML manually

UI/UX Specification#

Component Reference#

How to Use#

1. From Knowledge Base, click + New → Add Source, then choose Website.
2. Paste the full URL you want to import (including https://).
3. Optionally select a folder for organization.
4. Click Add to queue the page. The modal closes while the system fetches the content.
5. Refresh the knowledge base list after a moment to confirm the new source appears.

Notes & Tips#

• Ensure the page is publicly accessible; authenticated or paywalled URLs cannot be crawled.
• Use canonical URLs whenever possible so you do not ingest duplicate pages.
• If the page structure changes, re-add the URL to refresh the stored content.

FAQ#

Can I add multiple URLs at once?#

Add them sequentially. Each submission handles one URL to ensure the crawl succeeds.

How long does ingestion take?#

Simple pages usually import within a minute. Very large pages may take longer but appear in the queue shortly.

What if the crawl fails?#

You will see an error toast. Verify the URL is reachable, then try again or export the page manually and upload it as a document.