The team at AdminRemix built Chromebook Getter first as a Google App Script project but after having scaled now to over a one million installs, we learned very quickly that in order to ensure consistency across our product we had to own the infrastructure that Chromebook Getter relies on daily to ensure consistent user uploads.
When a G-Suite Admin Clicks Upload:
When we first started making uploads on a users behalf we would take the data in the current spreadsheet and send it off to Google’s servers and hope for the best. With time comes experience and now the process has changed largely for the better. When a user first initiates an upload of new ChromeOS device updates for Google, we do a couple error checks on their behalf before we even send the upload to our servers for processing, like ensuring that every device the user plans to upload has a valid device id & making sure that the new organizational units selected actually exist. If these checks do not pass we let the user know so they can make corrections. If initial checks pass we send the upload data to our api server in batches of 100 ChromeOS devices. Our API server then quickly normalize the data and pushes this data into a Redis work queue. Our consumer severs watching this queue see that there is work to be done and quickly send the data to Google’s servers on the users behalf, we ensure that all meta data is uploaded and all devices have been moved to the new organizational unit specified in the upload data. After everything goes right we update a job record in our database where the user is watching for completion to end the front-end app loading state.
How We Handle the Peak Scale:
We currently host our application with Heroku and use HireFire to handle auto scaling of our infrastructure, when a user or users send 1000’s of updates we quickly scale our worker servers to meet the new demand placed on us. Allowing all users regardless of size to get the same seamless experience. What this means as a user uploading 100K devices or 100 devices, the experience stays the same allowing for quick consistent uploads. Since moving away from Google App Script and to our own infrastructure we have seen upload times not only decrease by 85% but have received some amazing customer feedback.