Data Ocean concepts and terminology
The following concepts are used by Trimble Data Ocean for file management and object properties.
Access Control List (ACL)
Section titled “Access Control List (ACL)”This is an optional parameter that identifies read/write/delete access for a Data Ocean file. The owner of the file always has full access. ACL is not related to the restricted_access (Private/Public) setting.
Eventual consistency
Section titled “Eventual consistency”Parts of Data Ocean are eventually consistent, which means that, for example, a create file API request followed by a very quick get file API request can result in a 404 Object not found response and should be retried in this case if the expectation is that the data should be present. The data is eventually consistent usually within one second or less, therefore in most use cases eventual consistency will not affect behavior.
Fileset
Section titled “Fileset”A fileset is a collection of files managed through a single base object URL where Data Ocean does not manage the path of each individual file. For details on download paths, see Download files.
Multipart file
Section titled “Multipart file”For large files, use a multipart file upload to break it into parts. The file is reassembled in Data Ocean and accessed as a single object via download URL.
Object duration and expiration
Section titled “Object duration and expiration”The expiration of an object can be set in the file create request. Storage costs can be reduced by a defined expiration.
Ownership
Section titled “Ownership”In Data Ocean, the file is owned by the application that created the object.
Public/private object access
Section titled “Public/private object access”A file that is accessible through signed URLs. The parameter used for this is restricted_access. If this value is false, the file is publicly accessible.
Regions
Section titled “Regions”Data Ocean has built-in handling for geographic file distribution. A file can be replicated to and made available from multiple regions by setting its region preferences. For a list of regions, see List of AWS and Azure server regions.
Restricted access
Section titled “Restricted access”Signed URLs limit the permission and time that it takes to make a request. If the value for restricted_access is true then the URL will be signed. If this value is false, then the file will be publicly accessible.
Upload types
Section titled “Upload types”Data Ocean supports two types of uploads:
- Single Part
An upload performed in one part.
- Multipart
Multipart Upload allows you to upload a single object as a set of parts. This allows for larger file uploads. With this feature you can create parallel uploads. If one part fails to upload, it can be retried with no data loss of parts already uploaded.
URL Duration
Section titled “URL Duration”The validity of the URL duration, or expiration, is customizable and can be defined using url_duration. Security can be increased by decreasing the time a URL is valid.
Versions
Section titled “Versions”You can create different versions of a file object. When you associate files as new versions, they are treated as a set. If you delete a file object, all versions associated with that object are also deleted. Previous versions cannot be modified.
Reference Materials
Common errors using Data Ocean