Using classes for data access means that I can't easily do a SQL join between vendor data (which is always on the production DB) and data I generate (which might be on the dev DB. I don't love having business logic in SQL for the obvious reasons, but it can be very performant and easy to maintain. One advantage of DB clones is that it lets me write SQL that joins data I've generated on vendor data. This is a pain to set up, but works well. I've tried 2 solutions to this: One is have classes to handle all data reads and writes, and configuration that causes functions to read or write from a dev or production DB as needed. This ensures all teams are working on the same single source of data, improving the reliability of testing, supporting compliance, and making. For a backtest I might want to make a new database table, or change the behavior of an existing process that generates data. With whole-instance provisioning, every clone is a perfect copy, including database version, operating system, and configuration, resulting in controlled and standardized test and development data. But they are "dev-like" since I don't want my backtest to interfere with my production system, which generates the version of the model that I currently trade. My model is generated in several steps, which involves reading vendor data, generating output, then combining the output I generated in a previous step with more vendor data to generate the next step of output.īacktests are "prod-like" in that they must use real vendor data and must generate correct results that drive business decisions. To make this more complex, it's not a simple process where I read vendor data then generate analysis. The vendor data is too big to easily have two copies. The approach that you choose depends on your use case. Or you can share your Aurora DB cluster with another AWS account and let the account holder create the clone. You can create a clone of one of your Aurora DB clusters and share the clone. I can't run a backtest on a "dev" version of the vendor data, I need all of it to make my results correct. Such cross-account cloning is much faster than creating and restoring a database snapshot. Each backtest reads lots of vendor data from Postgres, and writes some output to Postgres. I generate "backtests" of a stock trading model, IE I see how it would have done historically.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |