cancel
Showing results for 
Search instead for 
Did you mean: 

Data Transformation Best Practices

josh
8 - Cloud Apps
8 - Cloud Apps

Where do you prefer to perform data transformations and why? Should it be done in a data warehouse, or is it best fit for a BI tool? I have a preferred answer, but I'm really interested to hear what customers and experts have to say!

2 REPLIES 2

shannon
8 - Cloud Apps
8 - Cloud Apps

It depends on the type of transformation -- for simpler transformations that aren't resource-intensive or live data transformations, I'll have the BI tool do the lifting. For complex transformations with multiple steps or transformations that are time or resource-intensive, I'll do the work on the warehouse end.

dougwalters
7 - Data Storage
7 - Data Storage

I try to think about who might need to update that transformation in the future - our data team has an engineer and an analyst. If it's something I only want the engineer to manage, we'll perform the transformation in Redshift. If it's something I want our analyst to manage, we'll perform the transformation in an Elasticube. That Redshift<>Elasticube connection point is where we delineate a lot of job responsibilities.