Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ include:

* **Automatic schema**. If a column is written that does not
exist on the table, it will be created automatically.
* **Upserts**. Records are either created or updated, depdending on
* **Upserts**. Records are either created or updated, depending on
whether an existing version can be found.
* **Query helpers** for simple queries such as all rows in a table or
all distinct values across a set of columns.
Expand Down Expand Up @@ -58,6 +58,14 @@ Here's the same example, but using the object-oriented API:
table.upsert(item, ['key1', 'key2'])


Here's an example showing a table being copied from one database to another:

from sqlaload import connect, create_table_from_existing

srcEngine = connect('mysql+mysqldb://username:password@111.111.111.111/database')
destEngine = connect('mysql+mysqldb://username:password@localhost/database')
create_table_from_existing(srcEngine, 'oldTable', destEngine, 'newTable')

Functions
---------

Expand All @@ -68,6 +76,7 @@ The library currently exposes the following functions:
* ``connect(url)``, connect to a database and return an ``engine``. See the [SQLAlchemy documentation](http://docs.sqlalchemy.org/en/rel_0_8/core/engines.html#database-urls) for information about URL schemes and formats.
* ``get_table(engine, table_name)`` will load a table configuration from the database, either reflecting the existing schema or creating a new table (with an ``id`` column).
* ``create_table(engine, table_name)`` and ``load_table(engine, table_name)`` are more explicit than ``get_table`` but allow the same functions.
* ``create_table_from_existing(src_engine, src_table_name, dest_engine, dest_table_name)`` will clone the schema and create a table from a table in another database. This is only likely to work if both the source database and destination database are the same type (mysql to postgresql won't work).
* ``drop_table(engine, table_name)`` will remove an existing table, deleting all of its contents.
* ``create_column(engine, table, column_name, type)`` adds a new column to a table, ``type`` must be a SQLAlchemy type class.
* ``create_index(engine, table, columns)`` creates an index on the given table, based on a list of strings to specify the included ``columns``.
Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@
include_package_data=False,
zip_safe=False,
install_requires=[
'sqlalchemy>=0.7',
'sqlalchemy-migrate>=0.7'
'sqlalchemy>=0.7'
],
tests_require=[],
entry_points=\
Expand Down
2 changes: 1 addition & 1 deletion sqlaload/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from sqlaload.schema import connect
from sqlaload.schema import create_table, load_table, get_table, drop_table
from sqlaload.schema import create_table, create_table_from_existing, load_table, get_table, drop_table
from sqlaload.schema import create_column
from sqlaload.write import add_row, update_row
from sqlaload.write import upsert, update, delete
Expand Down
20 changes: 15 additions & 5 deletions sqlaload/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from sqlalchemy import Integer, UnicodeText, Float, DateTime, Boolean
from sqlalchemy.schema import Table, MetaData, Column, Index
from sqlalchemy.sql import and_, expression
from migrate.versioning.util import construct_engine

log = logging.getLogger(__name__)
lock = RLock()
Expand All @@ -20,10 +19,7 @@ def connect(url):
from sqlalchemy.pool import NullPool
kw['poolclass'] = NullPool
engine = create_engine(url, **kw)
engine = construct_engine(engine)
meta = MetaData()
meta.bind = engine
engine._metadata = meta
engine._metadata = MetaData(bind=engine)
engine._tables = dict()
engine._indexes = dict()
return engine
Expand All @@ -38,6 +34,20 @@ def create_table(engine, table_name):
engine._tables[table_name] = table
return table

def create_table_from_existing(src_engine, src_table_name, dest_engine, dest_table_name):
with lock:
log.debug("Creating table: %s on %r" % (dest_table_name, dest_engine))
src_engine._metadata.reflect(src_engine)
srcTable = Table(src_table_name, src_engine._metadata)

destTable = Table(dest_table_name, dest_engine._metadata)
for column in srcTable.columns:
destTable.append_column(column.copy())

destTable.create()
dest_engine._tables[dest_table_name] = destTable
return destTable

def load_table(engine, table_name):
with lock:
log.debug("Loading table: %s on %r" % (table_name, engine))
Expand Down