Modern storage solutions are needed to store, access, and organize an ever-growing amount of data. Historically, application data gets stored on disk drives attached to servers running database software. While this architecture is optimal for data that is frequently updated, it’s hard to scale because your drives are attached to your database servers. Adding more storage can also require more compute capacity.
Object storage can solve the problem of scaling, and incorporates other features that are useful for a wide range of workloads, including customizable metadata, access control, versioning, and built-in replication.
What is Object Storage?
Object storage provides an abstraction layer for traditional storage devices. Data in an object storage system is stored and exposed as individual objects. Objects get updated through a REST API from any server or workstation. They also can be assigned optional unique URLs visited with an HTTP request.
Cloud-based object storage solutions pair well with hybrid or on-premise infrastructure. Because objects are accessed through an API, adding object storage does not require additional cloud-based compute products, which simplifies adoption. These solutions are also a natural match with existing cloud infrastructure and applications as the API is reached from the internet.
Finally, cloud-based object storage dynamically scales as you add and remove objects. In comparison, a block storage volume—which is the cloud analog of a traditional disk drive—has a fixed size, and resize operations are needed to shrink and grow them.
Use cases of object storage are diverse, ranging from simple multimedia storage to housing large datasets for machine learning. Earlier this year, we published an ebook on how to get the maximum value from object storage in the cloud. You can download “S3-Compatible Object Storage Use Cases” here (registration required). Here are a few examples we feature in the ebook.
Artifact Storage
As an increasing amount of development and an application’s life cycle become automated, more and more artifacts—like bulk collections of logs and version files—are generated from those processes. Discarding artifacts is not a best practice, as those residuals can often be beneficial later in the testing cycle. Developers and stakeholders can use object storage to save artifacts and share them in the production cycle by distributing their unique URLs.
Static Website Hosting
Object storage makes hosting a static website straightforward, with minimal maintenance required. Traditionally, static HTML files come from a web server (like NGINX or Apache) running on a virtual machine. This process is unnecessary with object storage, because you can make your objects publicly accessible over HTTP, and each will have a unique URL. Once you build your static site, you can make it available to the internet by uploading the files to an object storage bucket.
Using object storage to host your static site files gives you more time to focus on developing your website instead of routine server maintenance tasks like software upgrades, web server configuration, and security upkeep.
Unstructured Data
The need to store unstructured data was an original catalyst for the creation of object storage. Unstructured data is anything that does not fit into a traditional database, including log files, PDFs, receipts, maps, and multimedia files. What this data has in common—and what makes it unstructured—is that it lacks the internal relational information needed to fall neatly into a database column.
Object storage gives you the ability to configure custom metadata for each piece of unstructured data, making it easy to extrapolate useful information from each object and retrieve objects with similar metadata. Using other storage methods for unstructured data usually requires you to set up an additional database to organize the metadata separately.
Your unstructured data also gains an element of security with object storage. Both block storage and object storage are inherently private, but with one main difference: Block storage requires setting up a server to make data public on the internet. With object storage, all new objects in a bucket are set to private by default. To make objects public, you can use access controls—no server needed. Afterward, your objects will be accessible with HTTP requests.
Website Files
Object storage can help optimize your website or web application by hosting static assets like images, downloads, or user data in HTTP-accessible buckets. Whether building a dynamic website from scratch, using a content management system like WordPress, or an e-commerce platform like Magento, traditional file or block storage can carry a higher cost for storing website files. Object storage offers an alternative, cost-effective method to store and organize portions of your site or app’s assets.
If you need help adopting S3-Compatible Object Storage in your infrastructure, please let us know here or email proservices@linode.com
Comments