Photo by Andrew Neel on Unsplash

The New Model for Data Ops: Q&A with Rohit Choudhary, Founder & CEO at Acceldata

How have data and analytics evolved recently? What have been some of the most significant developments?

Data has also become much more operational. In the past, data was primarily used to populate monthly or quarterly reports to give people a snapshot of their businesses at a given point in time. Now, data powers a variety of real-time use cases that are core to business operations, and that is putting extra pressure on data teams to ensure the quality of data.

The time for democratization is now, especially when you consider the horizon. Within five years, everyone will have the same access to technology via the cloud — this includes everything from compute power to analytical and algorithmic capabilities. The whole curve is being flattened, which means data teams need the right capabilities to manage the technology that they adopt.

What should enterprises focus on to gain ground on data and analytics modernization initiatives?

Once you’ve decided to invest in a particular use case, it’s important to find the best technology with the largest community and fastest developing ecosystem. You also need to plan for unexpected operational concerns, which is why you should start thinking about data observability sooner rather than later.

How can businesses better plan and execute self-service analytics and data democratization for users who actually need it?

Leverage a data lake or data warehouse architecture to centralize as much data as possible, avoid data silos, and power more use cases with less complexity and administrative effort.

Why do data teams need data observability?

When something breaks or goes wrong, data engineers don’t have the context to understand the issue. It creates a compounding effect that places even more pressure on data teams to effectively monitor everything. This creates a great deal of work, and, without the right approach, this can quickly erode data engineering productivity — and the success of your data teams.

Data observability gives you end-to-end visibility into the health of your data and pipelines and gives you context to understand why things break or fail.

Developer productivity is the hidden secret to data success. How do you increase productivity? By keeping developers focused on business problems — as opposed to dealing with operational issues related to compute, data quality, or data pipelines. Data observability covers the surface area of the technology you’re implementing, saving developers time and effort and increasing their productivity.

Data observability also provides a common vocabulary to align your data science and analytics, operations, and engineering groups. At Acceldata, we accomplish this by providing a single pane of glass for data teams to manage their respective concerns while ensuring data is reliable, scalable, and optimized.

How do inefficiencies in the end-to-end orchestration of analytics pipelines affect organizations’ data scientists and engineers?

As with any journey, there can be unexpected detours and roadblocks along the way. Delays are particularly disruptive for data scientists who are waiting to apply value-added algorithms to fresh data sets. Delays create frustration for data engineers who are responsible for ensuring processes complete as expected. In either case, it would be nice to know what went wrong — and why.

Data observability provides end-to-end visibility across the entire data journey.

Can you share an insightful use case for Acceldata?

According to Burzin Engineer, Founder & Chief Reliability Officer at PhonePe, “Acceldata supports our hypergrowth and helps us manage one of the world’s largest instant payment systems. PhonePe’s biggest-ever data infrastructure initiative would never have been possible without Acceldata.”

What are the key trends driving the growth in data observability?

And, there simply aren’t enough engineers to support all of this growth. That’s why the top job in the US is data scientist / data engineer.

How do C-suite executives leverage data to deliver business value to their organizations?

What are you excited about looking at the immediate future? What is your larger vision for big data and data observability?

What advice would you give companies who are at the beginning of their digital transformation?

After all, data is intertwined with your digital transformation strategy.

How can one learn more about data observability?

This article was repurposed from The Time For Data Democratisation Is Now.



Thoughts and trends on data observability

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store