As part of our commitment to innovate and improve customer services, Autodesk released a new set of Data Management API endpoints to allow direct upload and download from the underlying AWS S3 bucket. The primary impact of this direct-to-S3 approach is performance.
Our testing and validation continue to show a significant increase in upload and download speeds, especially for large files.
How?
When an object is uploaded or downloaded from an OSS bucket, Autodesk uses AWS S3 storage.
The current binary transfer approach moves the data through our proxy, while the new direct-to-S3 approach enables you to directly access the underlying S3 bucket and object.
Direct binary streaming data management endpoints to be deprecated December 31, 2022
Autodesk will deprecate the current binary transfer streaming endpoints approach of uploading and downloading files directly to and from our proxy (
developer.api.autodesk.com
) on December 31, 2022, and such transfer will no longer be allowed after that date.
You will be required to move your existing application(s) utilizing impacted Autodesk APIs to the new direct-to-S3 approach by December 31, 2022.
See below for current information on impacted API endpoints and stay tuned for additional details to be released soon.
The following APIs are
affected
:
Data Management – Buckets & Objects
: The following endpoints will be removed:
PUT object
,
PUT resumable
, and
GET object
.
Applications should move to signeds3upload and signeds3download, which is explained in detail in the next section
(“Direct-to-S3 approach for Data Management OSS”)
.
Model Derivative
: The following endpoints will be removed:
GET Derivatives
Applications should move to the new endpoint
GET Fetch Derivative Download
Viewer:
If you are using any version prior to V7.68 you will need
Add or change the
endpoint
option when initializing the Viewer. No changes to other options are required
:
var options = {
endpoint: 'https://cdn.derivative.autodesk.com'
// keep other options unchanged
BIM 360 Docs PDF Export
-
GET export
endpoint is updated to provide additional S3 signed url, (i.e.,
signedUrl
field in the response). We can call this signed url to download the actual content. The current field,
link
, still functions at a moment for compatibility reason during migration period.
Read more about it here
The following APIs are
not affected, but your application may have the following cross dependencies
:
Design Automation
: OSS is commonly used as storage for the Design Automation workitems input and output.
Design Automation now improved the way to support the input/output url with the format like "urn:adsk.objects:os.object:<bucket_name>/<object_key>", developers are not required to generate the signed S3 url, what developer needs to prepare is just preparing the
objectId
and the access token, everything else will be automatically handled by Design Automation, and this new improvement is supported by all the Design Automation engines including Revit, 3ds Max, Inventor and AutoCAD.
Read more about it here.
Autodesk Construction Cloud & BIM360
: OSS is commonly used as storage for these services. No change is happening with them. As explained referred to the steps below.
Data Management – Projects, Folders & Files
: Applications that upload and download files will need to use signed S3 URLs.
Download:
Step 6
will change and require generating a signed S3 URL, then download using it.
Upload:
Step 4
will require 3 actions: generate signed S3 URL for upload, upload using the S3 URL, then call upload complete.
BIM 360/Autodesk Build Cost API -
As far as Cost API itself is concerned, there is
no change in Cost API
. This update is about instruction for upload/download workflows that have dependency to Data Management API, which is affected by the change. If you use Data Management API to upload/download files in Docs, you are most likely aware about the change already.
Read more about it here
The following APIs are
NOT affected
:
Authentication
Data Visualization
Token Flex
Webhooks
Assets
To upload and download files, applications must generate a signed URL, then upload or download the binary. Here are the steps (pseudo code):
Upload
Calculate the number of parts of the file to upload
Note: Each uploaded part except for the last one must be at least 5MB (1024 * 5)
Generate up to 25 URLs for uploading specific parts of the file using the
GET buckets/:bucketKey/objects/:objectKey/signeds3upload?firstPart=<index of first part>&parts=<number of parts>
endpoint
The part numbers start with 1
For example, to generate upload URLs for parts 10 through 15, set firstPart to 10 and parts to 6
This endpoint also returns an uploadKey that is used later to request additional URLs or to finalize the upload
Consider retrying (for example, with an exponential backoff) individual uploads when the response code is 100-199, 429, or 500-599
If the response code is 403, the upload URLs have expired; go back to step #2
If you have used up all the upload URLs and there are still parts that must be uploaded, go back to step #2
Finalize the upload using the
POST buckets/:bucketKey/objects/:objectKey/signeds3upload
endpoint, using the uploadKey value from step #2
Download
Generate a download URL using the
GET buckets/:bucketKey/objects/:objectName/signeds3download
endpoint
The signed URLs have a default expiration time of 2 minutes (*
longer expiration times can be set using the minutesExpiration param up to 60 minutes
). It is important that the download or upload process starts before the expiration. In case of single or multi-part upload, you may request signed URLs again. A set of steps for
uploading
and
downloading
can be found in our Step-by-Step Documentation.
In our next series of blog posts, we will be sharing with all of you a couple of samples the Developer Advocates are currently working on, keep an eye for the Node.js, .Netcore and .NET ones coming very soon.
Thank you again for all your business and please feel free to reach out to us through
[email protected]
.
*
DISCLAIMER When generating signed URLs, it’s important to use the smallest possible expiration time, in order to avoid longer access in case of exposure of the URL.
Tags:
