Manage geoDB Collections

The geoDB is a service provided by the Euro Data Cube project (EDC) as a paid service. It comes with a Python client that provides hugh level access to your data, and a certain amount of space in a PostGreSQL database. For managing your data you will need a management (read/write) account to your database which you can purchase at the EDC market place.

You can access the service in two ways:

  • By using the Jupyter Python notebook provided by EOX (configuration free, geodb = GeoDBClient())

  • By using you own Jupyter notebook or Python script by providing a client id and secret to the GeoDBClient (geodb = GeoDBClient(client_id="myid", client_secret="mysecet"))

The client ID and secret is also provided by EOX in the latter case. You will find them in your EOX hub account section. You can also provide the credentials via system environment variables (GEODB_AUTH_CLIENT_ID and GEODB_AUTH_CLIENT_SECRET). These variables can be supplied via a .env file.

There are two different types of geoDB accounts: a read only, and a management (read/write) access. The system will determine your access right through your authentication credentials.

Manage Collections in your GeoDB

[3]:

from xcube_geodb.core.geodb import GeoDBClient

Login from any maschine

Install xcube geoDB with command:

conda install xcube_geodb -c conda-forge

[1]:

### uncomment if not on EDC

#client_id=YourID
#client_secret=YourSecret
#geodb = GeoDBClient(client_id=client_id, client_secret=client_secret, auth_mode="client-credentials")

Login in EDC environment

[5]:

### comment if not on EDC

geodb = GeoDBClient()

Get your user name

[5]:

geodb.whoami

[5]:

'geodb_ci_test_user'

[6]:

# Lets get already existing collections
ds = geodb.get_my_collections()
ds


[6]:
owner database table_name
0 geodb_9bfgsdfg-453f-445b-a459 geodb_9bfgsdfg-453f-445b-a459 land_use
1 tt tt tt300

Creating collections

Once the connection has been established you will be able to create a collection. The collection will contain standard properties (fields) plus custom properties which you can add at your discretion. Please use PostGreSQL type definitions. We recommend stying simple with your data types as we have not tested every single type.

[7]:

# Have a look at fiona feature schema
collections = {
        "land_use":
        {
            "crs": 3794,
            "properties":
            {
                "RABA_PID": "float",
                "RABA_ID": "float",
                "D_OD": "date"
            }
        }
    }


geodb.create_collections(collections, clear=True)

[7]:

{'collections': {'geodb_ci_test_user_land_use': {'crs': 3794,
                                                 'properties': {'D_OD': 'date',
                                                                'RABA_ID': 'float',
                                                                'RABA_PID': 'float'}}}}

Loading data into a dataset

Once the table has been created, you can load data into the dataset. The example below loads a shapefile. The attributes of the shapefile correspond to the dataset’s properties.

[7]:

import geopandas
gdf = geopandas.read_file('data/sample/land_use.shp')
gdf

[7]:
RABA_PID RABA_ID D_OD geometry
0 4770326.0 1410 2019-03-26 POLYGON ((453952.629 91124.177, 453952.696 911...
1 4770325.0 1300 2019-03-26 POLYGON ((453810.376 91150.199, 453812.552 911...
2 2305689.0 7000 2019-02-25 POLYGON ((456099.635 97696.070, 456112.810 976...
3 2305596.0 1100 2019-02-25 POLYGON ((455929.405 97963.785, 455933.284 979...
4 2310160.0 1100 2019-03-11 POLYGON ((461561.512 96119.256, 461632.114 960...
... ... ... ... ...
9822 6253989.0 1600 2019-03-08 POLYGON ((460637.334 96865.891, 460647.927 969...
9823 6252044.0 1600 2019-03-26 POLYGON ((459467.868 96839.686, 459467.770 968...
9824 6245985.0 2000 2019-04-08 POLYGON ((459488.998 94066.248, 459498.145 940...
9825 6245986.0 2000 2019-02-20 POLYGON ((459676.680 94000.000, 459672.469 939...
9826 6245987.0 2000 2019-03-11 POLYGON ((459690.580 94042.607, 459686.872 940...

9827 rows × 4 columns

[8]:

geodb.insert_into_collection('land_use', gdf.iloc[:100,:]) # minimizing rows to 100, if you are in EDC, you dont need to make the subset.

[8]:

Data inserted into land_use

[10]:

geodb.get_collection('land_use', query="raba_id=eq.7000")

[10]:
id created_at modified_at geometry raba_pid raba_id d_od
0 3 2021-01-22T10:13:54.418035+00:00 None POLYGON ((456099.635 97696.070, 456112.810 976... 2305689 7000 2019-02-25
1 26 2021-01-22T10:13:54.418035+00:00 None POLYGON ((459898.930 100306.841, 459906.288 10... 2301992 7000 2019-04-06
2 95 2021-01-22T10:13:54.418035+00:00 None POLYGON ((459591.248 92619.056, 459592.745 926... 2333229 7000 2019-02-20

Delete from a Collection

[11]:

geodb.delete_from_collection('land_use', query="raba_id=eq.7000")

[11]:

Data from land_use deleted

[12]:

geodb.get_collection('land_use', query="raba_id=eq.7000")

[12]:
Empty Result

Updating a Collection

[13]:

geodb.get_collection('land_use', query="raba_id=eq.1300")

[13]:
id created_at modified_at geometry raba_pid raba_id d_od
0 2 2021-01-22T10:13:54.418035+00:00 None POLYGON ((453810.376 91150.199, 453812.552 911... 4770325 1300 2019-03-26
1 10 2021-01-22T10:13:54.418035+00:00 None POLYGON ((456547.427 91543.640, 456544.255 915... 2318555 1300 2019-03-14
2 63 2021-01-22T10:13:54.418035+00:00 None POLYGON ((456201.531 98685.274, 456199.109 986... 2304287 1300 2019-02-25
3 86 2021-01-22T10:13:54.418035+00:00 None POLYGON ((454709.766 97354.278, 454704.878 973... 2331038 1300 2019-01-05
4 87 2021-01-22T10:13:54.418035+00:00 None POLYGON ((453820.737 98574.017, 453816.740 985... 2357574 1300 2019-01-16
5 92 2021-01-22T10:13:54.418035+00:00 None POLYGON ((461723.552 99635.913, 461729.649 996... 2332405 1300 2019-03-27 
[14]:

geodb.update_collection('land_use', query="raba_id=eq.1300", values={'d_od': '2000-01-01'})

[14]:

land_use updated

[15]:

geodb.get_collection('land_use', query="raba_id=eq.1300")

[15]:
id created_at modified_at geometry raba_pid raba_id d_od
0 10 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((456547.427 91543.640, 456544.255 915... 2318555 1300 2000-01-01
1 86 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((454709.766 97354.278, 454704.878 973... 2331038 1300 2000-01-01
2 2 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((453810.376 91150.199, 453812.552 911... 4770325 1300 2000-01-01
3 63 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((456201.531 98685.274, 456199.109 986... 2304287 1300 2000-01-01
4 87 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((453820.737 98574.017, 453816.740 985... 2357574 1300 2000-01-01
5 92 2021-01-22T10:13:54.418035+00:00 2021-01-22T10:15:11.329375+00:00 POLYGON ((461723.552 99635.913, 461729.649 996... 2332405 1300 2000-01-01

Managing Properties of a Collection

[16]:

geodb.get_my_collections()

[16]:
owner database table_name
0 geodb_9bfgsdfg-453f-445b-a459 geodb_9bfgsdfg-453f-445b-a459 land_use
1 geodb_ci_test_user geodb_ci_test_user land_use
2 tt tt tt300 
[17]:

geodb.get_properties('land_use')

[17]:
database table_name column_name data_type
0 geodb_ci_test_user land_use id integer
1 geodb_ci_test_user land_use created_at timestamp with time zone
2 geodb_ci_test_user land_use modified_at timestamp with time zone
3 geodb_ci_test_user land_use geometry USER-DEFINED
4 geodb_ci_test_user land_use raba_pid double precision
5 geodb_ci_test_user land_use raba_id double precision
6 geodb_ci_test_user land_use d_od date 
[18]:

geodb.add_property('land_use', "test_prop", 'integer')

[18]:

Properties added

[19]:

geodb.get_properties('land_use')

[19]:
database table_name column_name data_type
0 geodb_ci_test_user land_use id integer
1 geodb_ci_test_user land_use created_at timestamp with time zone
2 geodb_ci_test_user land_use modified_at timestamp with time zone
3 geodb_ci_test_user land_use geometry USER-DEFINED
4 geodb_ci_test_user land_use raba_pid double precision
5 geodb_ci_test_user land_use raba_id double precision
6 geodb_ci_test_user land_use d_od date
7 geodb_ci_test_user land_use test_prop integer 
[20]:

geodb.drop_property('land_use', 'test_prop')

[20]:

Properties ['test_prop'] dropped from geodb_ci_test_user_land_use

[21]:

geodb.get_properties('land_use')

[21]:
database table_name column_name data_type
0 geodb_ci_test_user land_use id integer
1 geodb_ci_test_user land_use created_at timestamp with time zone
2 geodb_ci_test_user land_use modified_at timestamp with time zone
3 geodb_ci_test_user land_use geometry USER-DEFINED
4 geodb_ci_test_user land_use raba_pid double precision
5 geodb_ci_test_user land_use raba_id double precision
6 geodb_ci_test_user land_use d_od date 
[22]:

geodb.add_properties('land_use', properties={'test1': 'integer', 'test2': 'date'})

[22]:

Properties added

[23]:

geodb.get_properties('land_use')

[23]:
database table_name column_name data_type
0 geodb_ci_test_user land_use id integer
1 geodb_ci_test_user land_use created_at timestamp with time zone
2 geodb_ci_test_user land_use modified_at timestamp with time zone
3 geodb_ci_test_user land_use geometry USER-DEFINED
4 geodb_ci_test_user land_use raba_pid double precision
5 geodb_ci_test_user land_use raba_id double precision
6 geodb_ci_test_user land_use d_od date
7 geodb_ci_test_user land_use test1 integer
8 geodb_ci_test_user land_use test2 date 
[24]:

geodb.drop_properties('land_use', properties=['test1', 'test2'])

[24]:

Properties ['test1', 'test2'] dropped from geodb_ci_test_user_land_use

[8]:

geodb.get_properties('land_use')

[8]:
table_name column_name data_type
0 geodb_admin_land_use id integer
1 geodb_admin_land_use created_at timestamp with time zone
2 geodb_admin_land_use modified_at timestamp with time zone
3 geodb_admin_land_use geometry USER-DEFINED
4 geodb_admin_land_use raba_pid double precision
5 geodb_admin_land_use raba_id double precision
6 geodb_admin_land_use d_od date
7 geodb_admin_land_use testä_prop integer 
[25]:

geodb.drop_collection('land_use')

[25]:

Collection ['geodb_ci_test_user_land_use'] deleted

[ ]: