EuroPython 2016

SQLAlchemy as the backbone of a Data Science company

Speaker(s) Peter Hoffmann

In times of NoSQL databases and Map Reduce Algorithms it’s surprising how far you can scale the relational data model. At Blue Yonder we use SQLAlchemy in all stages of our data science workflows and handle tenth of billions of records to feed our predictive algorithms. This talk will dive into SQLAlchemy beyond the Object Relational Mapping (ORM) parts and conentrate on the SQLAlchemy Core API and the Expression Language:

  • Database Abstraction: Statements are generated properly for different database vendor and type without you having to think about it.

  • Security: Database input is escaped and sanitized prior to beeing commited to the database. This prevents against common SQL injection attacks.

  • Composability and Reuse: Common building blocks of queries are expressed as SQLAlchemy selectables and can be reuesd in other queries.

  • Testability: SQLAlchemy allows you to perform functional tests against a database or mock out queries and connections.

  • Reflection: Reflection is a technique that allows you to generate a SQLAlchemy repesentation from an existing database. You can reflect tables, views, indexes, and foreign keys.

As a result of the usage of SQLAlchemy in Blue Yonder, we have implemented and open sourced a SQLAlchemy dialect for the in memory, column-oriented database system EXASolution

in on Wednesday 20 July at 11:15 See schedule

Do you have some questions on this talk?

New comment