How these two data roles are similar but very different
When I started as a data engineer, I worked on a team focused on DevOps. Even though it wasn’t exactly what I wanted to do in my first role, it taught me a lot. Looking back, if I hadn’t been in this type of role back then, I probably wouldn’t have the experience I have as an analytical engineer today.
Now, working as an analytics engineer, I focus on something called DataOps. Although this may sound similar to DevOps, they are very different. DevOps focuses on software as a product while DataOps focuses on producing high-quality data. For those focused on DataOps, data is the product!
While working as a DevOps Data Engineer, I helped software engineers make changes to our web application code. I focused on testing UI changes after each deployment rather than the specifics of the data. Not once did I check the number of rows in a table or if the values of a field were populated. Instead, I made sure no errors were thrown on the backend.
As an analytics engineer, every time I make a change to code or put something into production, I need to focus on metadata or data about data. This involves writing validation queries to ensure that things like the number of rows, number of columns, and distribution of values look like they did before I made a change. Or, if I want them to be different from before, they reflect those changes!
Although DevOps and DataOps seem similar, they serve two different purposes. In this article, we’ll dig deeper into the differences, covering the product they aim to serve and the different indicators of success.
DevOps involves deploying and testing changes to software code. When I worked as a DevOps engineer, it often involved long nights deploying, testing code changes in many different environments, and validating the changes with the software engineers who made them.