Scan

A Scan is the comparison object in the Scanerr application. A Scan is a snapshot representation of a Page's state at a given time. Scans contains all the data used in the Test comparison algorithm. A Scan has 5 primary components:

html -> a page-source dump of the page's web page
logs -> a list of all present JavaScript and Network errors and warining
lighthouse -> an object containing Lighthouseopen in new window scores and audits
yellowlab -> an object containing Yellow Lab toolsopen in new window scores and audits
images -> a list of image objects (screenshots spanning lenght of webpage)

Scan object

{
   "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",       // uuid specific to the Scan
   "site":"9fb29910-c88b-4b1e-a226-3baf2fb57a3c",     // uuid specific to the Site
   "page":"9fb35910-bg8b-3bwe-a426-2fve53b56a3b",     // uuid specific to the Page
   "paired_scan":"",                                  // uuid of Scan being compared to - the Test endpoints configure this
   "time_created":"2021-11-18T17:01:33.998970Z",      // timestamp of object creation
   "time_completed":"2021-11-18T17:01:35.637830Z",    // timestamp of object completion (if null - task is still working)
   "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],    // an array containing the types of data to collect
   "tags": ["tag1", "tag2"],                                      // <optional> array of info for user to track data
   "configs" : {                                   // Configurations for webdriver
      "driver": "selenium",                        // webdriver type - one of selenium or puppeteer
      "device": "desktop",                         // sets the user-Agent, one of dekstop or mobile
      "mask_ids": "example-id-1, example-id-2",    // element id's you wish to mask when taking a screenshot (seperated by comma)
      "interval": 5,                               // time (seconds) the driver will wait between checking if page has loaded
      "window_size": "1920,1080",                  // dimensions of webdriver window. format -> (width,height)
      "max_wait_time": 60,                         // maximum time (seconds) the driver will wait before moving on to next task
      "min_wait_time": 10                          // minimum time (seconds) the driver will wait before checking if page has loaded and or moving on to next task
   }
   "html":"<html lang=\"en\" dir=\"ltr\"><head><script id=\"ipV6 ", // page source  of Site  *SHORTENED FOR DISPLAY PURPOSES
   "logs":[                                                         // an array of found console errors and warnings
      {
         "level":"WARNING",
         "source":"javascript",
         "message":"https://www.example.com/?toWww=1&redig=B407E2FB9E444333B0C3170A134D1997 - The resource https://www.example.com/rp/lmu8EBCaPRMKtay8LSArGyY3mv4.br.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.",
         "timestamp":1637254867884
      }
   ],
   "lighthouse": {
      "scores":{   // high level Lighthouse scores
         "seo":85,    
         "average":78.25,  
         "performance":60, 
         "accessibility":75,   
         "best_practices":93,
         "pwa": 89, 
         "crux": 90 
      },
      "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",
      // json file containing all utilized audits for each of the Lighthouse categories      
      // for more info on Lighthouse outputs visit --> https://github.com/GoogleChrome/lighthouse/blob/master/docs/understanding-results.md
   },
   "yellowlab": {  // highlevel Yelllow Lab scores 
      "scores": {
         "fonts":40,
         "badCSS":20,
         "jQuery":100,
         "images":93,
         "pageWeight":82,
         "globalScore":72,
         "serverConfig":67,
         "badJavascript":88,
         "cssComplexity":13,
         "domComplexity":100,
         "javascriptComplexity":41
      },
      "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",
      // json file containing all utilized audits for each of the Yellow Lab categories
      // for more info on Yellow Lab outputs visit --> https://github.com/YellowLabTools/YellowLabTools  
   },
   "images": [  // an ordered array of screenshot images 
      {
         "id":"d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf",    // uuid specific to the image
         "url":"https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png",     // exact url path to image on s3 storage
         "path":"static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png",     // path to image on s3 storage - used by boto3 library
         "index":0                                       // position of image in array - used in test comparison 
      },
      // SHORTENED FOR DISPLAY PURPOSES
   ]
}

Create a Scan

There are two endpoints for creating a site. The most commonly used is the delay endpoint which allows the creation task to run asynchronously on the server and resolves quickly. Alternatively, you may use the traditional endpoint which may take several minutes to resolve as the server is actively creating a scan and will return the entire Scan object.

`data` in this request:

"data": {
   "site_id": "<site:id>",          // <required> 
   
   "tags": ["tag1", "tag2"],        // <optional> for user to track data
   "configs": {                     // <optional> Configurations for webdriver 
      "driver": "selenium",         // webdriver type - one of selenium or puppeteer
      "device": "desktop",          // sets the user-Agent, one of dekstop or mobile
      "mask_ids": null,             // element id's you wish to mask when taking a screenshot (seperated by comma)
      "interval": 5,                // time (seconds) the driver will wait between checking if page has loaded
      "window_size": "1920,1080",   // dimensions of webdriver window. format -> (width,height)
      "max_wait_time": 60,          // maximum time (seconds) the driver will wait before moving on to next task
      "min_wait_time": 10           // minimum time (seconds) the driver will wait before checking if page has loaded and or moving on to next task
   }
}

WARNING

Passing the configs object is optional. However, if you do pass the object you must specify all sub-components.

`POST - /scan/delay`

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{SCANERR_API_BASE_URL}/scan/delay'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}
data = {
    "site_id": "<site:id>",
    "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],
    "tags": ["tag1", "tag2"], # <optional> for user to track data
    "configs": {
      "driver": "selenium",
      "device": "desktop",
      "mask_ids": None,
      "interval": 5,
      "window_size": "1920,1080",
      "max_wait_time": 60,
      "min_wait_time": 10
    }
}

# send the request
res = requests.post(
    url=url,
    headers=headers,
    data=json.dumps(data)
)

# retrieve response data
json_response = res.json()
print(json_response)

Output:


{  
   "status": true,
   "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",
   "message": "scan is being created in the background"
}

Retrieve a Scan

This endpoint returns a single Scan object and is useful as a simple "detailed view" of the scan.

`GET - /scan/<scan:id>`

import requests, os

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{BASE_URL}/scan/<scan:id>'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}

# send the request
res = requests.get(
    url=url,
    headers=headers,
)

# retrieve response data
json_response = res.json()
print(json_response)

View Full Output

Output:

{
   "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",
   "site":"9fb29910-c88b-4b1e-a226-3baf2fb57a3c",
   "page":"9fb35910-bg8b-3bwe-a426-2fve53b56a3b",
   "paired_scan":"",
   "time_created":"2021-11-18T17:01:33.998970Z",
   "time_completed":"2021-11-18T17:01:35.637830Z",
   "html":"<html lang=\"en\" dir=\"ltr\"><head><script id=\"ipV6 " # SHORTENED FOR DISPLAY PURPOSES
   "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],
   "tags": ["tag1", "tag2"],
   "logs":[
      {
         "level":"WARNING",
         "source":"javascript",
         "message":"https://www.example.com/?toWww=1&redig=B407E2FB9E444333B0C3170A134D1997 - The resource https://www.example.com/rp/lmu8EBCaPRMKtay8LSArGyY3mv4.br.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.",
         "timestamp": 1637254867884
      }
   ],
   "lighthouse": {
      "scores":{
         "seo": 82,
         "average": 78.0,
         "performance": 50,
         "accessibility": 93,
         "best_practices": 87 ,
         "pwa": 89, 
         "crux": 90
      },
      "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",    
   },
   "yellowlab": { 
      "scores": {
         "fonts": 40,
         "badCSS": 20,
         "jQuery": 100,
         "images": 93,
         "pageWeight": 82,
         "globalScore": 72,
         "serverConfig": 67,
         "badJavascript": 88,
         "cssComplexity": 13,
         "domComplexity": 100,
         "javascriptComplexity": 41
      },
      "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",
   },
   "images": [
      {
         "id":"d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf",  
         "url":"https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png", 
         "path":"static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png",     
         "index": 0  
      },
      # SHORTENED FOR DISPLAY PURPOSES
   ]
}

Retrieve a lean Scan

This endpoint returns an abbreviated version of a single Scan object. Omits the larger portions of the Scan payload including html, logs, images, and audits for Lighthouse and Yellowlab.

`GET - /scan/<scan:id>/lean`

import requests, os

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{BASE_URL}/scan/<scan:id>/lean'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}

# send the request
res = requests.get(
    url=url,
    headers=headers,
)

# retrieve response data
json_response = res.json()
print(json_response)

View Full Output

Output:

{
   "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",
   "site":"9fb29910-c88b-4b1e-a226-3baf2fb57a3c",
   "page":"9fb35910-bg8b-3bwe-a426-2fve53b56a3b",
   "paired_scan":"",
   "time_created":"2021-11-18T17:01:33.998970Z",
   "time_completed":"2021-11-18T17:01:58.234551Z",
   "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],
   "tags": ["tag1", "tag2"],
   "lighthouse": {
      "scores":{
         "seo": 82,
         "average": 78.0,
         "performance": 50,
         "accessibility": 93,
         "best_practices": 87 ,
         "pwa": 89, 
         "crux": 90
      },
   },
   "yellowlab": { 
      "scores": {
         "fonts": 40,
         "badCSS": 20,
         "jQuery": 100,
         "images": 93,
         "pageWeight": 82,
         "globalScore": 72,
         "serverConfig": 67,
         "badJavascript": 88,
         "cssComplexity": 13,
         "domComplexity": 100,
         "javascriptComplexity": 41
      },
   },
}

Retrieve many Scans

This endpoint returns a paginated response with all Scan objects filtered by the passed page_id and ordered by time_created. This endpoint is useful when needing to displaying your sites in a table view for example. The limit parameter specifies the total number of objects you want returned per "group" (we recomend keeping this under 10 for best performance). The offset parameter specfies which "group" to return. For example, limit=10&offset=10 in a total dataset of 30 objects would return 10 scans in range scan #10 - scan #20.

TIP

Additionally, an optional parameter is page_id=<page:id> which would limit the objects filtering to the associated site.

`GET - /scan?page_id=<page:id>&limit=10&offset=0`

import requests, os

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{BASE_URL}/scan?page_id=<page:id>&limit=10&offset=0'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}

# send the request
res = requests.get(
    url=url,
    headers=headers,
)

# retrieve response data
json_response = res.json()
print(json_response)

View Full Output

Output:

{
   "count":30,
   "next":"https://api.scanerr.io/v1/ops/scan?limit=10&offset=20",
   "previous":"https://api.scanerr.io/v1/ops/scan?limit=10&offset=10", 
   "results":[
      {
         "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",
         "site":"9fb29910-c88b-4b1e-a226-3baf2fb57a3c",
         "page":"9fb35910-bg8b-3bwe-a426-2fve53b56a3b",
         "paired_scan":"",
         "time_created":"2021-11-18T17:01:33.998970Z",
         "time_completed":"2021-11-18T17:01:35.637830Z",
         "html":"<html lang=\"en\" dir=\"ltr\"><head><script id=\"ipV6 " # SHORTENED FOR DISPLAY PURPOSES
         "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],
         "tags": ["tag1", "tag2"],
         "logs":[
            {
               "level":"WARNING",
               "source":"javascript",
               "message":"https://www.example.com/?toWww=1&redig=B407E2FB9E444333B0C3170A134D1997 - The resource https://www.example.com/rp/lmu8EBCaPRMKtay8LSArGyY3mv4.br.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.",
               "timestamp": 1637254867884
            }
         ],
         "lighthouse": {
            "scores":{
               "seo": 82,
               "average": 78.0,
               "performance": 50,
               "accessibility": 93,
               "best_practices": 87 ,
               "pwa": 89, 
               "crux": 90
            },
            "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",    
         },
         "yellowlab": { 
            "scores": {
               "fonts": 40,
               "badCSS": 20,
               "jQuery": 100,
               "images": 93,
               "pageWeight": 82,
               "globalScore": 72,
               "serverConfig": 67,
               "badJavascript": 88,
               "cssComplexity": 13,
               "domComplexity": 100,
               "javascriptComplexity": 41
            },
            "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",
         },
         "images": [
            {
               "id":"d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf",  
               "url":"https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png", 
               "path":"static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png",     
               "index": 0  
            },
         ]
      },
      {
         "id":"b2af59fe-17ad-4a91-b1d8-6bd80e23b8a2",
         "site":"9fb29910-c88b-4b1e-a226-3baf2fb57a3c",
         "page":"9fb35910-bg8b-3bwe-a426-2fve53b56a3b",
         "paired_scan":"",
         "time_created":"2021-11-18T17:01:33.998970Z",
         "time_completed":"2021-11-18T17:01:35.637830Z",
         "html":"<html lang=\"en\" dir=\"ltr\"><head><script id=\"ipV6 " # SHORTENED FOR DISPLAY PURPOSES
         "type": ["html", "logs", "lighthouse", "yellowlab", "vrt"],
         "tags": ["tag1", "tag2"],
         "logs":[
            {
               "level":"WARNING",
               "source":"javascript",
               "message":"https://www.example.com/?toWww=1&redig=B407E2FB9E444333B0C3170A134D1997 - The resource https://www.example.com/rp/lmu8EBCaPRMKtay8LSArGyY3mv4.br.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.",
               "timestamp": 1637254867884
            }
         ],
         "lighthouse": {
            "scores":{
               "seo": 82,
               "average": 78.0,
               "performance": 50,
               "accessibility": 93,
               "best_practices": 87 ,
               "pwa": 89, 
               "crux": 90
            },
            "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",    
         },
         "yellowlab": { 
            "scores": {
               "fonts": 40,
               "badCSS": 20,
               "jQuery": 100,
               "images": 93,
               "pageWeight": 82,
               "globalScore": 72,
               "serverConfig": 67,
               "badJavascript": 88,
               "cssComplexity": 13,
               "domComplexity": 100,
               "javascriptComplexity": 41
            },
            "audits": "https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/d14b67cc-0073-42e7-a3a7-e721c74dfcb1/0c5b30af-0197-4b68-8b2a-417c61142948/ffc603bc-5711-456f-94c1-71832dec6ed3/51624df5-3649-4baa-a788-f059606f92ac.json",
         },
         "images": [
            {
               "id":"d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf",  
               "url":"https://storage-scanerr.sfo3.digitaloceanspaces.com/static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png", 
               "path":"static/sites/ba601adb-1bd6-471f-9c30-a03e5242f40b/d6ea99e6-bdb5-4a4a-8f6d-a659d51863bf.png",     
               "index": 0  
            },
         ]
      },
      
      ### SHORTENED FOR DISPLAY PURPOSES ###

   ]
}

Delete a Scan

Because Scans have a primary link to Tests, it is advisable to only delete them if you also plan to delete the associated Tests.

DANGER

Please use caution with this endpoint as it is irreversible.

`DELETE - /scan/<scan:id>`

import requests, os

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{BASE_URL}/scan/<scan:id>'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}

# send the request
res = requests.delete(
   url=url, 
   headers=headers, 
)

# retrieve response data
json_response = res.json()
print(json_response)

View Full Output

Output:

    {
      "message": "scan deleted successfully"
    }

Delete many Scans

This endpoint allows the user to send an array of Scan id's to be deleted. Scanerr will iterate through the id's and respond with an object detailing which Scans were successfully deleted and which were not.

DANGER

Please use caution with this endpoint as it is irreversible.

`POST - /scans/delete`

import requests, os

# import env vars
SCANERR_API_BASE_URL = os.environ.get('SCANERR_API_BASE_URL')
SCANERR_API_TOKEN = os.environ.get('SCANERR_API_BASE_URL')

# setup configs
url = f'{BASE_URL}/scans/delete'
headers = {
    "content-type": "application/json",
    "Authorization" : SCANERR_API_TOKEN
}

data = {
   "ids": ['<scan:id1>', '<scan:id2>', '<scan:id3>']
}

# send the request
res = requests.post(
    url=url, 
    headers=headers, 
    data=data,
)

# retrieve response data
json_response = res.json()
print(json_response)

View Full Output

Output:

    {
      "status": false,
      "num_succeeded": 2,
      "succeeded": [ 
         '<scan:id2>', 
         '<scan:id3>'
      ],
      "num_failed": 1,
      "failed": [
         '<scan:id1>',
      ]
    }

# Scan

# Scan object

# Create a Scan

# data in this request:

# POST - /scan/delay

# Output:

# Retrieve a Scan

# GET - /scan/<scan:id>

# Output:

# Retrieve a lean Scan

# GET - /scan/<scan:id>/lean

# Output:

# Retrieve many Scans

# GET - /scan?page_id=<page:id>&limit=10&offset=0

# Output:

# Delete a Scan

# DELETE - /scan/<scan:id>

# Output:

# Delete many Scans

# POST - /scans/delete

# Output:

Scan

Scan object

Create a Scan

`data` in this request:

`POST - /scan/delay`

Output:

Retrieve a Scan

`GET - /scan/<scan:id>`

Output:

Retrieve a lean Scan

`GET - /scan/<scan:id>/lean`

Output:

Retrieve many Scans

`GET - /scan?page_id=<page:id>&limit=10&offset=0`

Output:

Delete a Scan

`DELETE - /scan/<scan:id>`

Output:

Delete many Scans

`POST - /scans/delete`

Output: