Amazon Web Services vs Cloudera

2022. Cloudera launches its all-in-one SaaS data lakehouse



Cloudera, the company that specializes in big data with a focus on Hadoop, is now shifting its focus towards becoming the unified data fabric for hybrid data platforms. Taking a step further in this direction, the company recently launched its Cloudera Data Platform (CDP) One, a data lakehouse as a service (LaaS). This managed offering aims to provide enterprises with a platform that enables self-service analytics and data access for a broader range of employees. While Databricks, known for popularizing the lakehouse concept, also offers SaaS-based solutions, Cloudera positions its service as the "first all-in-one data lakehouse SaaS offering." Cloudera emphasizes that its service combines compute, storage, machine learning, streaming analytics, and enterprise security, making it a comprehensive solution for organizations.


2020. AWS launches Amazon AppFlow, its new SaaS integration service


AWS has recently launched Amazon AppFlow, an integration service designed to simplify data transfer between AWS and popular SaaS applications such as Google Analytics, Marketo, Salesforce, ServiceNow, Slack, Snowflake, and Zendesk. Similar to competing services like Microsoft Azure's Power Automate, developers can configure AppFlow to trigger data transfers based on specific events, predetermined schedules, or on-demand requests. Unlike some competitors, AWS positions AppFlow primarily as a data transfer service rather than an automation workflow tool. While the data flow can be bidirectional, AWS's emphasis is on moving data from SaaS applications to other AWS services for further analysis. To facilitate this, AppFlow includes various tools for data transformation as it passes through the service.


2019. AWS launches fully-managed backup service for business


Amazon's cloud platform, AWS, has introduced a new service called Backup, allowing companies to securely back up their data from various AWS services as well as their on-premises applications. For on-premises data backup, businesses can utilize the AWS Storage Gateway. This service enables users to define backup policies and retention periods according to their specific requirements. It includes options such as transferring backups to cold storage for EFS data or deleting them entirely after a specified duration. By default, the data is stored in Amazon S3 buckets. While most of the supported services already offer snapshot creation capabilities (except for EFS file systems), Backup automates this process and adds customizable rules to enhance data protection. Notably, the pricing for Backup aligns with the costs associated with using the snapshot features (except for file system backup, which incurs a per-GB charge).


2018. Big Data platforms Cloudera and Hortonworks merge



Over time, Hadoop, the once-prominent open-source platform, fostered the growth of numerous companies and an ecosystem of vendors. However, the complexity associated with Hadoop posed a significant challenge. This is where companies like Hortonworks and Cloudera stepped in, offering packaged solutions for IT departments seeking the advantages of a big data processing platform without the need to build Hadoop from scratch. These companies provided various approaches to tackle the complexity, but as cloud-based big data solutions gained prominence, the notion of implementing a Hadoop system from scratch became less compelling, even with the assistance of firms like Cloudera and Hortonworks. Today, both companies have announced their merger in a deal valued at $5.2 billion. The combined entity will serve a customer base of 2,500, generate $720 million in revenue, and possess $500 million in cash reserves, all while remaining debt-free.


2017. AWS launched browser-based IDE for cloud developers



Amazon Web Services has introduced a new browser-based Integrated Development Environment (IDE) called AWS Cloud9. While it shares similarities with other IDEs and editors like Sublime Text, AWS emphasizes its collaborative editing capabilities and deep integration into the AWS ecosystem. The IDE includes built-in support for various programming languages such as JavaScript, Python, PHP, and more. Cloud9 also provides pre-installed debugging tools. AWS positions this as the first "cloud native" IDE, although competitors may contest this claim. Regardless, Cloud9 offers seamless integration with AWS, enabling developers to create cloud environments and launch new instances directly from the tool.


2017. AWS introduced per-second billing for EC2 instances



In recent years, several alternative cloud platforms have shifted towards more flexible billing models, primarily adopting per-minute billing. However, AWS is taking it a step further by introducing per-second billing for its Linux-based EC2 instances. This new billing model applies to on-demand, reserved, and spot instances, as well as provisioned storage for EBS volumes. Furthermore, both Amazon EMR and AWS Batch are transitioning to this per-second billing structure. It is important to note that there is a minimum charge of one minute per instance, and this change does not affect Windows machines or certain Linux distributions that have their own separate hourly charges.


2017. AWS offers a virtual machine with over 4TB of memory



Amazon's AWS has introduced its largest EC2 machine yet, the x1e.32xlarge instance, boasting an impressive 4.19TB of RAM. This represents a significant upgrade from the previous largest EC2 instance, which offered just over 2TB of memory. These machines are equipped with quad-socket Intel Xeon processors operating at 2.3 GHz, up to 25 Gbps of network bandwidth, and two 1,920GB SSDs. It is evident that only a select few applications require this level of memory capacity. Consequently, these instances have obtained certification for running SAP's HANA in-memory database and its associated tools, with SAP offering direct support for deploying these applications on these instances. It's worth mentioning that Microsoft Azure's largest memory-optimized machine currently reaches just over 2TB of RAM, while Google's maximum memory capacity caps at 416GB.


2015. Hortonworks acquired dataflow solutions developer Onyara



Hortonworks, a publicly traded company that offers a commercial distribution of the open-source big data software Hadoop, has announced its acquisition of Onyara, an early-stage startup known for the development of Apache NiFi. This open-source software originated within the National Security Agency (NSA) and enables efficient delivery of sensor data to appropriate systems while maintaining data tracking capabilities. In addition to previous acquisitions like XA Secure and SequenceIQ, Hortonworks has now expanded its portfolio with the intention of introducing a new subscription service based on Apache NiFi. This subscription will be marketed under the name Hortonworks DataFlow.


2015. Google partners with Cloudera to bring Cloud Dataflow to Apache Spark



Google has announced a collaboration with Cloudera, the Hadoop specialists, to integrate its Cloud Dataflow programming model into Apache's Spark data processing engine. By bringing Cloud Dataflow to Spark, developers gain the ability to create and monitor data processing pipelines without the need to manage the underlying data processing cluster. This service originated from Google's internal tools for processing large datasets at a massive scale on the internet. However, not all data processing tasks are identical, and sometimes it becomes necessary to run tasks in different environments such as the cloud, on-premises, or on various processing engines. With Cloud Dataflow, data analysts can utilize the same system to create pipelines, regardless of the underlying architecture they choose to deploy them on.


2014. AWS now supports Docker containers



Amazon has announced the preview availability of EC2 Container Services, a new service dedicated to managing Docker containers and enhancing the support for hybrid cloud in Amazon Web Services. This offering brings forth a range of benefits, including streamlined development management, seamless portability across different environments, reduced deployment risks, smoother maintenance and management of application components, and comprehensive interoperability. It is important to note that AWS is not the first cloud provider to provide support for Docker's open-source engine. Google has recently expanded its support for Docker containers through the introduction of its Google Container Engine, which is powered by its own Kubernetes and was announced just last week during the Google Cloud Platform Live event. Furthermore, Microsoft had previously announced its support for Kubernetes in managing Docker containers in Azure back in August.