This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Cookie Policy. Click Preferences to customize your cookie settings.
Where do you prefer to perform data transformations and why? Should it be done in a data warehouse, or is it best fit for a BI tool? I have a preferred answer, but I'm really interested to hear what customers and experts have to say!
It depends on the type of transformation -- for simpler transformations that aren't resource-intensive or live data transformations, I'll have the BI tool do the lifting. For complex transformations with multiple steps or transformations that are time or resource-intensive, I'll do the work on the warehouse end.
I try to think about who might need to update that transformation in the future - our data team has an engineer and an analyst. If it's something I only want the engineer to manage, we'll perform the transformation in Redshift. If it's something I want our analyst to manage, we'll perform the transformation in an Elasticube. That Redshift<>Elasticube connection point is where we delineate a lot of job responsibilities.
