URL Lists
Custom URL lists offer flexibility to supersede the predefined Netskope URL category mapping for a given URL and/or augment them by defining custom URL categories for situations in which the Netskope predefined URL category does not have a mapping for a URL (uncategorized).
Before you begin, compile a list of URLs to include in a policy scan, and if needed, create another list of URLs to exclude from a policy scan. You can enter the URLs individually in the UI, create a CSV file with all the URLs, or use the REST API V2. Once you complete this step, you must create a custom category to apply in a policy.
Important
URL List enhancements are applicable for Real-time Protection policies, SSL Decryption policies, steering exceptions, and REST API V2. Contact Support to enable these enhancements in your account.
To add a URL list:
Go to Policies > Web > URL Lists.
To add a new URL list, click New URL List.
Enter a name for your URL list.
In the URL & IP Address enter the URLs or IP addresses separated by a new line, or upload a CSV file (max upload is 8 MB). You can also define a port number for URLs.
Note
Support for IP addresses is in Controlled GA. If you want to enable this feature in your tenant, contact your sales team.
When adding URLs, ensure you are following the formatting rules:
URL Format
Matched URLs
www.example.com/path/to/resource
www.example.com/path/to/resource/a
www.example.com/path/to/resource/a/b
example.com
example.com/
example.com/path/a
www.example.com
www.example.com
www.example.com/path/a
www.example.com/some/path
www.example.com/query?a=b
http://www.example.com
https://www.example.com
http://www.example.com:80
https://www.example.com:443
*.example.com
example.com
www.example.com
www.us.example.com
www.uk.example.com
www.example.com:8080
http://www.example.com:8080
https://www.example.com:8080
www.example.com:80
http://www.example.com
https://www.example.com:80
http://www.example.com:80
www.example.com:443
https://www.example.com
https://www.example.com:443
http://www.example.com:443
www.example.com/path/*/
Not Supported
www.example.com/*
www.example.com/path/*
Not Supported
Wildcard domains (e.g., *.example.com) include the root domain and all subdomains. Keep in mind that if two or more wildcard entries, subdomains, and paths are present in your URL list, then the Netskope service uses the longest entry for categorization.
Note
The configuration to allow your URL lists to match against all matching URLs (instead of only the longest prefix match) is in Controlled GA. If you want to enable this behavior in your tenant, contact your sales team.
Before disabling the longest prefix match behavior, Netskope recommends reviewing your policies thoroughly as this is a fundamental change to how Netskope processes your policies in SWG.
You can add IP addresses ranges and IP addresses with CIDR notation.
Tip
When creating a URL list for destination IP addresses, consider the following:
When you create a Real-time Protection policy for destination IP addresses, you can also create an SSL Do Not Decrypt policy. However, if the Real-time Protection policy’s action is Block, then the activity will be blocked without SSL interception. In other words, the block page will not appear for users when they access the destination IP addresses.
To block the activity with SSL interception, you must create a Real-time Protection policy only without an SSL policy.
When you create a URL list for destination IP addresses, the No SNI error setting option is unavailable. This is because when a user accesses a destination IP address, there’s no SNI.
To learn more about adding URLs and IP addresses, see URL List Best Practices.
Click Save.
Click Apply Changes.
Note
The REST API can be used to add to an existing URL list, but not create a new URL list. For more information, refer to Update a URL List.
Optionally, if you have URL List enhancements enabled in your account, you will see the following window and options. Select URL Type > Exact Match or Regex. Admins can use regex-based URL matching in the URL List. See URL List Best Practices for the regex syntac guidelines. The default behavior is exact match URLs.
Type the URLs separated by a new line (not comma separated) or upload a CSV file (max upload is 8 MB) and click Next.
Enter a name for your URL list and click Save URL List.
Click Apply Changes.
API Endpoints
The following API endpoints are available with REST API V2 for URL lists:
Note
REST API V2 is part of the URL List enhancements, contact Support to enable this feature in your account.
Endpoint | Description |
|---|---|
GET /api/v2/policy/urllist | Gets all applied and pending URL lists |
POST /api/v2/policy/urllist | Creates a new URL list |
POST /api/v2/policy/urllist/file | Uploads multiple configurations via a JSON file |
GET /api/v2/policy/urllist/{id} | Gets URL list by ID |
PUT /api/v2/policy/urllist/{id} | Replaces a URL list configuration |
DELETE /api/v2/policy/urllist/{id} | Deletes a URL list |
PATCH /api/v2/policy/urllist/{id}/{action} | Patches a URL list |
POST /api/v2/policy/urllist/deploy | Applies a pending URL list of changes |
Example:
You can query the URL list followed by a field from a specific URL list. To query specific fields use the following query:
GET http://netskope.betaskope.com/api/v2/policy/urllist?field=id,name
Available field / query parameters include:
Id
name
data
pending
modify_time
modify_type
modify_by