Transfer data between databases with PostgreSQL

Question

I need to transfer some data from another database. The old database is called paw1.moviesDB and the new database is paw1. The schema of each table are the following.

Awards (name of the table)(new DB)
Id [PK] Serial           Award

Nominations (name of the table) (old DB)
Id [PK] Serial           nominations

How do I copy the data from old database to the new database?

Answer 1

I just had to do this exact thing so I figured I'd post the recipe here. This assumes that both databases are on the same server.

First, copy the table from the old db to the new db. At the commandline:

pg_dump -U postgres -t <old_table> <old_database> | psql -U postgres -d <new_database>

Next, grant permissions of the copied table to the user of the new database. Log into psql:

psql -U postgres -d <new_database>

ALTER TABLE <old_table> OWNER TO <new_user>;

\q

At this point your copied table in your new database still has the name <old_table> from your old database. Assuming you want to move the data somewhere else, say to <new_table>, you can just use regular SQL queries:

INSERT INTO <new_table> (field1, field2, field3) 
SELECT field1, field2, field3 from <old_table>;

Done!

Answer 2

Databases are isolated in PostgreSQL; when you connect to a PostgreSQL server you connect to just one database, you can't copy data from one database to another using a SQL query.

If you come from MySQL: what MySQL calls (loosely) "databases" are "schemas" in PostgreSQL - sort of namespaces. A PostgreSQL database can have many schemas, each one with its tables and views, and you can copy from one schema to another with the schema.table syntax.

If you really have two distinct PostgreSQL databases, the common way of transferring data from one to another would be to export your tables (with pg_dump -t ) to a file, and import them into the other database (with psql).

If you really need to get data from a distinct PostgreSQL database, another option - mentioned in Grant Johnson's answer - is dblink, which is an additional module (in contrib/).

Update:

Postgres introduced "foreign data wrapper" in 9.1 (which was released after the question was asked). Foreign data wrappers allow the creation of foreign tables through the Postgres FDW which makes it possible to access a remote table (on a different server and database) as if it was a local table.

Answer 3

This worked for me to copy a table remotely from my localhost to Heroku's postgresql:

pg_dump -C -t source_table -h localhost source_db | psql -h destination_host -U destination_user -p destination_port destination_db

This creates the table for you.

For the other direction (from Heroku to local) pg_dump -C -t source_table -h source_host -U source_user -p source_port source_db | psql -h localhost destination_db

Answer 4

From: hxxp://dbaspot.c om/postgresql/348627-pg_dump-t-give-where-condition.html (NOTE: the link is now broken)

# create temp table with the data
psql mydb
CREATE TABLE temp1 (LIKE mytable);
INSERT INTO temp1 SELECT * FROM mytable WHERE myconditions;
\q

# export the data to a sql file
pg_dump --data-only --column-inserts -t temp1 mtdb > out.sql
psql mydb
DROP TABLE temp1;
\q

# import temp1 rows in another database
cat out.sql | psql -d [other_db]
psql other_db
INSERT INTO mytable (SELECT * FROM temp1);
DROP TABLE temp1;

Another method useful in remotes

  # export a table csv and import in another database
  psql-remote> COPY elements TO '/tmp/elements.csv' DELIMITER ',' CSV HEADER;
  $ scp host.com:/tmp/elements.csv /tmp/elements.csv
  psql-local> COPY elements FROM '/tmp/elements.csv' DELIMITER ',' CSV;

Answer 5

There are three options for copying it if this is a one off:

1. Use a db_link (I think it is still in contrib)
2. Have the application do the work.
3. Export/import

If this is an ongoing need, the answers are:

1. Change to schemas in the same DB
2. db_link

Answer 6

1. If your source and target database resides in the same local machine, you can use:

Note:- Sourcedb already exists in your database.

CREATE DATABASE targetdb WITH TEMPLATE sourcedb;

This statement copies the sourcedb to the targetdb.

2. If your source and target databases resides on different servers, you can use following steps:

Step 1:- Dump the source database to a file.

pg_dump -U postgres -O sourcedb sourcedb.sql

Note:- Here postgres is the username so change the name accordingly.

Step 2:- Copy the dump file to the remote server.

Step 3:- Create a new database in the remote server

CREATE DATABASE targetdb;

Step 4:- Restore the dump file on the remote server

psql -U postgres -d targetdb -f sourcedb.sql

(pg_dump is a standalone application (i.e., something you run in a shell/command-line) and not an Postgres/SQL command.)

This should do it.

Answer 7

You can not perform a cross-database query like SQL Server; PostgreSQL does not support this.

The DbLink extension of PostgreSQL is used to connect one database to another database. You have install and configure DbLink to execute a cross-database query.

I have already created a step-by-step script and example for executing cross database query in PostgreSQL. Please visit this post: PostgreSQL [Video]: Cross Database Queries using the DbLink Extension

Answer 8

Actually, there is some possibility to send a table data from one PostgreSQL database to another. I use the procedural language plperlu (unsafe Perl procedural language) for it.

Description (all was done on a Linux server):

1. Create plperlu language in your database A

2. Then PostgreSQL can join some Perl modules through series of the following commands at the end of postgresql.conf for the database A:

   plperl.on_init='use DBI;'
   plperl.on_init='use DBD::Pg;'
   

3. You build a function in A like this:

   CREATE OR REPLACE FUNCTION send_data( VARCHAR )
   RETURNS character varying AS
   $BODY$
   my $command = $_[0] || die 'No SQL command!';
   my $connection_string =
   "dbi:Pg:dbname=your_dbase;host=192.168.1.2;port=5432;";
   $dbh = DBI->connect($connection_string,'user','pass',
   {AutoCommit=>0,RaiseError=>1,PrintError=>1,pg_enable_utf8=>1,}
   );
   my $sql = $dbh-> prepare( $command );
   eval { $sql-> execute() };
   my $error = $dbh-> state;
   $sql-> finish;
   if ( $error ) { $dbh-> rollback() } else {  $dbh-> commit() }
   $dbh-> disconnect();
   $BODY$
   LANGUAGE plperlu VOLATILE;
   

And then you can call the function inside database A:

SELECT send_data( 'INSERT INTO jm (jm) VALUES (''zzzzzz'')' );

And the value "zzzzzz" will be added into table "jm" in database B.

Answer 9

Just like leonbloy suggested, using two schemas in a database is the way to go. Suppose a source schema (old DB) and a target schema (new DB), you can try something like this (you should consider column names, types, etc.):

INSERT INTO target.Awards SELECT * FROM source.Nominations;

Answer 10

I think that use of the pg_dump utility can be restricted by a PostgreSQL Server admin.

So I used \copy to meta commands to export to CSV and import into destination database.