BrickLink Access

Database rebuild

The hourly job that (re)builds BrickStore's database uses a dedicated BrickLink account to login (from GitHub servers, unless debugging is required) and then download the following:

• Mostly XML, but since the additional categories for items are missing from the XML, BrickStore has to download the items as CSV as well:

  • https://www.bricklink.com/catalogDownload.asp

• Just plain text:

  • https://www.bricklink.com/btinvlist.asp
  • https://www.bricklink.com/btchglog.asp

• As the relationship data is not available as a XML/CSV download, these pages have to be HTML-scraped at the moment:

  • https://www.bricklink.com/catalogRelList.asp
  • https://www.bricklink.com/catalogRelCat.asp

BrickStore itself

As for the main BrickStore app that runs on the users machine, there are 3 types of requests: unauthenticated, unauthenticated with API key and authenticated.

1. Unauthenticated requests are used to download item pictures:

   • https://img.bricklink.com/ItemImage/[itemTypeId]/N/[colorId]/[itemId].png

2. Unauthenticated with API key requests are used for price-guide data:

   • https://api.bricklink.com/api/affiliate/v1/price_guide_batch

     (HTML post with with query parameter "api_key")

3. Authenticated is used for all other requests: store inventory, order lists, shopping carts, wanted lists:

   • Store inventory: The simplest of them all - plain XML:

     • https://www.bricklink.com/invExcelFinal.asp

   • Orders: The order list and items are downloaded as XML

     • https://www.bricklink.com/orderExcelFinal.asp

     Because the XML is missing buyer/seller address information, this page has to be HTML scraped as well:

     • https://www.bricklink.com/orderDetail.asp

   • Shopping carts: The list is kind of HTML scrapped: download the HTML, search for the JS instruction var GlobalCart = { ... }; and then parse the contents inside {} as JSON.

     • https://www.bricklink.com/v2/globalcart.page

     The actual shopping cart contents are JSON already:

     • /ajax/renovate/cart/getStoreCart.ajax

   • Wanted lists: This works similar to the shopping carts: the global list comes from a JSON snippet inside an HTML page (var wlJson):

     • https://www.bricklink.com/v2/wanted/list.page

     The actual shopping cart contents are standard XML, downloaded from:

     • https://www.bricklink.com/files/clone/wanted/downloadXML.file

Authentication

As for the authentication itself, BrickStore simulates a web browser:

1. It logs in via:

   • https://www.bricklink.com/ajax/renovate/loginandout.ajax

2. It then tries to keep the login session alive by requesting both of these every 30min:

   • https://www.bricklink.com/ajax/renovate/SessionInfoGet.ajax
   • https://www.bricklink.com/refresh.asp

3. If any request returns a 302 redirection to v2/login/page or login.asp?, BrickStore will try to log in via (1.) again and then resend the request.

4. If the user changes their credentials in BrickStore, and BrickStore thinks it is currently logged in, it will log out via /ajax/renovate/loginandout.ajax first, before logging in via (1.) again.

5. The first login is done lazily. This means that BrickStore will not log into BrickLink, unless an user action requires it.