Database management

Complicated and Competitive: How Vertica Navigates Today’s Complex World of Database Management

There is a lot of tension in the world of database management, and it is not for the faint of heart.

Vertica Systems Inc. has played a high stakes poker game that, over its 15 year history, has involved the world’s three largest public cloud providers, critical decisions of large companies regarding data migration. outside of on-site networks, and two major presidential campaigns. Big bets have big consequences, and that’s where the tension comes in.

Vertica, now part of Micro Focus International PLC, launched its technology in 2005 to address the scalability and performance issues of traditional enterprise data warehouses, a year before Amazon.com Inc. launched its public cloud.

As Amazon’s cloud has grown into a disruptive corporate colossus, it has also developed its own cloud-native Redshift database. This created competition, mainly from Snowflake Inc., and Vertica was left with a key decision to make. Should it join the cloud native part or remain firmly anchored in the on-premises world?

The answer came in 2018 when Vertica introduced Eon mode to support running on Amazon EC2 compute and S3 storage. It was the first data store that enabled compute and storage independent scaling for cloud and on-premises workloads.

To make things more interesting, Snowflake and Vertica quickly made deals with Amazon Web Services Inc., Microsoft Corp. and Google LLC to integrate services, although the three major cloud providers had their own in-house data warehousing platforms.

AWS had Redshift, Microsoft ran Azure SQL Data Warehouse, and Google offered BigQuery. It created tension, which Snowflake CEO Frank Slootman readily acknowledged.

“It’s inevitable for us to challenge these first-generation, proprietary cloud products,” Slootman said, in an interview with Diginomica at the company’s Summit event in October. “Does this create tension? Yes. We are competing against them, and that creates a very complex dynamic and depending on the day of the week things can get a little difficult. “

Cloud native vs hybrid

Vertica must not only compete with Snowflake, it must also fight for market share against the products of its own partners. This has put the business at the heart of a debate within the corporate IT community as to whether data-driven workloads should stay on-premises, migrate in bulk to the cloud, or run in both. ?

Recent survey data from the Cloud Native Computing Foundation shows signs of moving towards a hybrid solution. While 62% preferred the public cloud, this was down from 77% the previous year. Today, 38% prefer a hybrid model, and the CNCF has said it “fully expects this trend to continue.”

Fortunately for Vertica, its decision to launch Eon Mode two years ago allowed the company to capitalize on the growing interest in the hybrid. The technology’s ability to separate compute from storage has proven invaluable for businesses that depend on rapid elasticity to cope with changing workloads.

A good example of this can be found at The Trade Desk, an online ad placement company that bids hundreds of thousands of ad auctions per second, reaching billions of users across the globe. The Trade Desk broke Rackspace Inc.’s cloud twice and was more than happy to do the same for Eon mode when Vertica was beta testing it.

“Vertica said, ‘Are you ready to break it for us? And we say to ourselves’ No problem! Can do! “Dave Pickles, CTO and co-founder of The Trade Desk said in an interview last year. “We broke 25 versions of it. But they kept on composing it and it went really well at the end, so we’re really happy with how it turned out. “

Beyond integration

Breaking systems over and over again can create its own special form of tension, but there is one method to Vertica’s madness. With Eon Mode, the company recognizes that developers want to go beyond building apps that simply run efficiently on a cloud platform.

Real value comes when solutions are so ingrained in the design process that developers can take full advantage of the range of cloud services on offer. Integration is not enough. The database management solution should be an architecture designed from the ground up to operate with maximum efficiency in the cloud.

This critical distinction was underscored by Ben Vandiver, former CTO of Vertica and main responsible for the design and deployment of Eon Mode.

“With Eon Mode, Vertica is moving from simple integration with cloud services to introducing a base architecture optimized specifically for the cloud so that customers can capitalize on the economics of separating compute and storage.” Vandiver said in an interview in 2018.

Separating compute and storage was a key advancement for Vertica as it provided enterprise customers with granular scalability. If they ran out of storage, customers didn’t have to buy unnecessary compute to fix the problem.

Eon Mode was the first data store that enabled compute and storage independent scaling in on-premises and cloud environments, but Snowflake provided it from the start in cloud-native environments, all like Google with BigQuery and Amazon with its RA3 nodes for Redshift.

Campaign issues

This competitive tension between partners spilled over into an unlikely but important space: national politics. In 2012, Vertica was instrumental in the success of President Barack Obama’s re-election effort when he was selected as the campaign’s primary analytical database management system.

However, according to a description published in Wired, the data operation for Hillary Clinton’s 2016 campaign encountered numerous IT issues, some of which were attributed by DNC staff to Vertica. In 2019, the Democratic National Committee replaced Vertica with a database driven by Google’s BigQuery.

A person familiar with the Democratic campaign operation in 2016, told SiliconANGLE that the party did not significantly patch or upgrade its hardware after Obama’s re-election in 2012. Vertica’s software had to run on a system. which had not been upgraded for four years, according to this source.

The problems were also compounded by the departure of many key figures from the IT team that led the operation of the database for Obama in 2012, according to a trusted source intimately familiar with Project Vertica at the time. , who spoke to SiliconANGLE.

Vertica’s experience highlights a simple reality: It’s not just businesses that need database management technology in a cloud-driven world. AWS now operates the websites of the Federal Election Commission, Republican National Committee, and DNC, in addition to providing services to state and country election offices in more than 40 states.

There is a lot of tension around data these days. Is it useful or irrelevant, protected or not, too little or too much? Vertica finds herself in the eye of the hurricane, and there is no other place her top leader would rather be.

“If you look at the disruption that happens with data, companies that compete with data aren’t just outdoing everyone else in their space, in some cases they’re reinventing and opening up massive new opportunities,” he said. said Colin Mahony, senior vice president of Vertica. Chairman and CEO, during an interview in December. “Vertica is really at the heart of this disruption. “

Image: Pixabay

Show your support for our mission by joining our community of Cube Club and Cube Event experts. Join the community which includes Amazon Web Services and Amazon.com CEO Andy Jassy, ​​Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many other luminaries and experts.


Source link