[Coming Soon] Search & Crawler Management Settings

4. EC + SC (EN).png

As customers increasingly use AI for product searches, SHOPLINE has launched the "Search & Crawler Management" feature. It provides a one-stop interface for merchants to edit brand highlights, best-sellers, and more, helping AI better understand and represent your brand in customer interactions.

In this article, you'll find:

 

1. Set up in SHOPLINE Admin 

Step 1

Go to [Marketing & Tracker] > [Search & Crawler Management] and select the "AI Search Optimization" tab.

SHOPLINE Admin "Search & Crawler Management" page, showing three steps under the "AI Search Optimization" tab: Edit AI-Readable Content, Preview AI-Readable Content, Verify AI Answers

 

Step 2

Click the Edit button in the "Edit AI-Readable Content" section.

SHOPLINE Admin Search & Crawler Management feature, Edit AI-Readable Content field

 

Step 3

Once on the editing page, you will see system-generated default content. You can edit the file in the "Edit Text" section to add brand highlights, best-selling products, or custom AI response guidelines.

*Notes:

  • Please use brief summaries or bullet points for better AI understanding. 
  • You can enter up to 10,000 characters. Further input is restricted.
SHOPLINE Admin Search & Crawler Management feature, example of editing AI-readable content

 

Editing example

Add descriptions of featured products in the product section, then click Save at the top right.

SHOPLINE Admin Search & Crawler Management feature, clicking Save after editing AI-readable content

 

Once saved, a "Successfully updated" notification will appear, and it will direct you back to the "Search & Crawler Management" page.

SHOPLINE Admin Search & Crawler Management feature, "Successfully updated" popup after editing AI-readable content

 

*Note: To revert to the system-generated default content, click Reset to default at the top right. After confirming in the pop-up, the content will be updated, and it will direct you back to the "Search & Crawler Management" page.

SHOPLINE Admin Search & Crawler Management feature, "Reset to Default" button in AI-readable content editing

 

Step 4

Once completed, click View llms.txt file in the "Preview AI-Readable Content" section to preview the content.

SHOPLINE Admin Search & Crawler Management feature, Preview AI-Readable Content section

 

*Example of llms.txt file: Below is the system-generated default content, which includes the store introduction, product recommendations, and other information.

SHOPLINE Admin Search & Crawler Management feature, example of default-generated llms.txt file

 

Step 5

Follow the instructions in the "Verify AI Answers" section, then enter your usage scenarios or questions into an AI search tool. Check whether the AI can correctly introduce your brand and products, or answer questions about store policies.

SHOPLINE Admin Search & Crawler Management feature, Verify AI Answers section

 

Verification example

The llms.txt file guides AI to determine stock status using visual cues like button text, and includes structured data so AI can cross-reference stock markers in the webpage source code.

SHOPLINE Admin Search & Crawler Management feature, example of verifying AI answers

 

Using an AI search tool (Gemini is used as an example below), ask whether a specific product is in stock or available for pre-order.

SHOPLINE Admin Search & Crawler Management feature, entering a question in the AI answer verification example

 

After AI generates a response, compare it with the information on your storefront or Admin to verify its accuracy. If further adjustments are needed, you can update your llms.txt file in the "Edit AI-Readable Content" section.

SHOPLINE Admin Search & Crawler Management feature, comparing AI answer against storefront in the verification example

*Notes:

  • AI crawlers may take a few hours to a few days to crawl data. 
  • While llms.txt helps guide AI search accuracy, AI models rely on probabilistic reasoning, so outputs may vary and cannot be fully controlled.

 

2. Feature permissions management

Step 1

In the SHOPLINE Admin, go to [Settings] > [Permission & Security]. Click Edit to adjust the staff member's permission settings.

SHOPLINE Admin Search & Crawler Management feature, permission management

 

Step 2

In the "Marketing & Tracker" section under "Admin Access Rights", find "Search & Crawler Management" and set the staff member's view and add/edit permissions as needed.

SHOPLINE Admin Search & Crawler Management feature, setting view or edit permission fields

 

3. Search engine files 

In addition to setting up the "AI Search Optimization" feature, you can also view the following files in the "Search Engine Files" tab.

SHOPLINE Admin Search & Crawler Management feature, Site Crawling Documents tab

 

i. Search engine access policy

The system has automatically generated two files, llms.txt and llms-full.txt, and added their locations to robots.txt and the homepage to help AI crawlers locate them. Click View robots.txt file to check the file content.

SHOPLINE Admin, Search Engine Access Rules, robots.txt file

 

ii. Sitemap

The system automatically includes all multilingual URLs displayed on your storefront in sitemap.xml. Click View sitemap.xml file to see the sitemap index for each language.

SHOPLINE Admin Sitemap, multilingual URL index

 

4. Notes

  • llms.txt and llms-full.txt currently share the same content.
  • If the llms.txt file uses the system default content, there will be a cache delay of about 12 hours before a store name update is reflected in the file.
  • If the same file is edited simultaneously in multiple windows, only the last saved version will be kept.

 

 

Read more

 

 

 

 

 

Have more questions? Submit a request

Comments