Sentinelsat Readthedocs Io en v0.12.2
Sentinelsat Readthedocs Io en v0.12.2
Release 0.12.2
1 Contents 3
1.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Python API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Known errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.5 Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.6 Indices and tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
i
ii
Sentinelsat Documentation, Release 0.12.2
Sentinelsat makes searching, downloading and retrieving the metadata of Sentinel satellite images from the Copernicus
Open Access Hub easy.
It offers an easy-to-use command line interface
Contents 1
Sentinelsat Documentation, Release 0.12.2
2 Contents
CHAPTER 1
Contents
1.1 Installation
1.1.1 Tests
By default, prerecorded responses to Copernicus Open Access Hub queries are used to not be affected by its downtime.
To allow the tests to run actual queries against Copernicus Open Access Hub set the environment variables
and add --vcr disable to py.test arguments. To update the recordings use either --vcr record_new or
--vcr reset.
Sentinelsat has been tested with Python versions 2.7 and 3.4+. Earlier Python 3 versions are expected to work as well
as long as the dependencies are fulfilled.
3
Sentinelsat Documentation, Release 0.12.2
The convenience functions to_dataframe() and to_geodataframe() require pandas and/or geopandas
to be present.
Sentinelsat provides a CLI sentinelsat to query and download multiple or single images.
1.2.1 Quickstart
A basic search query consists of a search polygon as well as the username and password to access the Copernicus
Open Access Hub.
For convenience and added security you can set the username, password and DHuS URL as environment variables and
omit their entry on the command line.
export DHUS_USER="<user>"
export DHUS_PASSWORD="<password>"
export DHUS_URL="<api_url>"
sentinelsat -g <geojson>
Search areas are provided as GeoJSON polygons, which can be created with QGIS or geojson.io. If you do not specify
a start and end date only products published in the last 24 hours will be queried.
Start and end dates refer to the acquisition date given by the beginPosition of the products, i.e. the start of the
acquisition time.
Sentinel-1
Search and download all Sentinel-1 scenes of type SLC over a search polygon, in descending orbit for the year 2015.
Download a single Sentinel-1 GRDH scene covering Santa Claus Village in Finland on Christmas Eve 2015.
4 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
Sentinel-2
Search and download Sentinel-2 scenes for January 2016 with a maximum cloud cover of 40%.
1.2.2 sentinelsat
Options:
ESA maintains a list of valid search keywords that can be used with --query.
The options --sentinel, --instrument and --producttype are mutually exclusive and follow a hierarchy
from most specific to least specific, i.e. --producttype > --instrument > --sentinel. Only the most
specific option will be included in the search when multiple ones are given.
Searching by name supports wildcards, such as S1A_IW*20151224* to find all Sentinel-1 A scenes from 24th of
December 2015 without restricting the result to a search area.
1.3.1 Quickstart
# GeoPandas GeoDataFrame with the metadata of the scenes and the footprints as
˓→geometries
api.to_geodataframe(products)
# Get basic information about the product: its title, file size, MD5 sum, date,
˓→footprint and
Valid search query keywords can be found at the Copernicus Open Access Hub documentation.
In addition to the search query keywords sentinelsat allows filtering and sorting of search results before download.
To simplify these operations sentinelsat offers the convenience functions to_geojson(), to_dataframe() and
to_geodataframe() which return the search results as a GeoJSON object, Pandas DataFrame or a GeoPandas
GeoDataFrame, respectively. to_dataframe() and to_geodataframe() require pandas and geopandas
to be installed, respectively.
6 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
In this example we query Sentinel-2 scenes over a location and convert the query results to a Pandas DataFrame. The
DataFrame is then sorted by cloud cover and ingestion date. We limit the query to first 5 results within our timespan
and download them, starting with the least cloudy scene. Filtering can be done with all data types, as long as you pass
the id to the download function.
products_df_sorted = products_df_sorted.head(5)
Sentinelsat provides two methods for retrieving product metadata from the server, one for each API offered by the
Copernicus Open Access Hub:
• query() for OpenSearch (Solr), which supports filtering products by their attributes and returns metadata for
all matched products at once.
• get_product_odata() for OData, which can be queried one product at a time but provides the full meta-
data available for each product, as well as information about the product file such as the file size and checksum,
which are not available from OpenSearch.
Both methods return a dictionary containing the metadata items. More specifically, query() returns a dictionary
with an entry for each returned product with its ID as the key and the attributes’ dictionary as the value.
All of the attributes returned by the OpenSearch API have a corresponding but differently named attribute in the
OData’s full metadata response. See the DataHubSystem’s metadata definition files to find the exact mapping be-
tween them (OpenSearch attributes have a <solrField> tag added): - Sentinel-1 attributes - Sentinel-2 attributes -
Sentinel-3 attributes
OpenSearch example
'format': 'SAFE',
'gmlfootprint': '<gml:Polygon srsName="http://www.opengis.net/gml/srs/
˓→epsg.xml#4326" xmlns:gml="http://www.opengis.net/gml">\n <gml:outerBoundaryIs>\n
˓→ <gml:LinearRing>\n <gml:coordinates>0.401648,34.322010 0.876987,36.
˓→540989 -0.747357,36.884121 -1.227940,34.664474 0.401648,34.322010</gml:coordinates>
'link_alternative': "https://scihub.copernicus.eu/apihub/odata/v1/
˓→Products('04548172-c64a-418f-8e83-7a4d148adf1e')/",
'link_icon': "https://scihub.copernicus.eu/apihub/odata/v1/Products(
˓→'04548172-c64a-418f-8e83-7a4d148adf1e')/Products('Quicklook')/$value",
'missiondatatakeid': 110481,
'orbitdirection': 'ASCENDING',
'orbitnumber': 16302,
'platformidentifier': '2014-016A',
'platformname': 'Sentinel-1',
'polarisationmode': 'VV VH',
'productclass': 'S',
'producttype': 'SLC',
'relativeorbitnumber': 130,
'sensoroperationalmode': 'IW',
'size': '7.1 GB',
'slicenumber': 8,
'status': 'ARCHIVED',
'summary': 'Date: 2017-04-25T15:56:12.814Z, Instrument: SAR-C SAR,
˓→Mode: VV VH, Satellite: Sentinel-1, Size: 7.1 GB',
'uuid': '04548172-c64a-418f-8e83-7a4d148adf1e'}),
...
OData example
Only the most basic information available from the OData API is returned by default, if full=True is not set. The
full metadata query response is quite large and not always nrequired, so it is not requested by default.
>>> api.get_product_odata('04548172-c64a-418f-8e83-7a4d148adf1e')
{'date': datetime.datetime(2017, 4, 25, 15, 56, 12, 814000),
'footprint': 'POLYGON((34.322010 0.401648,36.540989 0.876987,36.884121 -0.747357,34.
˓→664474 -1.227940,34.322010 0.401648))',
'id': '04548172-c64a-418f-8e83-7a4d148adf1e',
'md5': 'E5855D1C974171D33EE4BC08B9D221AE',
(continues on next page)
8 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
With full=True we receive the full metadata available for the product.
>>> api.get_product_odata('04548172-c64a-418f-8e83-7a4d148adf1e', full=True)
{'Acquisition Type': 'NOMINAL',
'Carrier rocket': 'Soyuz',
'Cycle number': 107,
'Date': datetime.datetime(2017, 4, 25, 15, 56, 12, 814000),
'Filename': 'S1A_IW_SLC__1SDV_20170425T155612_20170425T155639_016302_01AF91_46FF.SAFE
˓→',
˓→</gml:LinearRing>\n </gml:outerBoundaryIs>\n</gml:Polygon>',
'Format': 'SAFE',
'Identifier': 'S1A_IW_SLC__1SDV_20170425T155612_20170425T155639_016302_01AF91_46FF',
'Ingestion Date': datetime.datetime(2017, 4, 25, 19, 23, 45, 956000),
'Instrument': 'SAR-C',
'Instrument abbreviation': 'SAR-C SAR',
'Instrument description': '<a target="_blank" href="https://sentinel.esa.int/web/
˓→sentinel/missions/sentinel-1">https://sentinel.esa.int/web/sentinel/missions/
˓→sentinel-1</a>',
'Instrument description text': 'The SAR Antenna Subsystem (SAS) is developed and
˓→build by AstriumGmbH. It is a large foldable planar phased array antenna, which
˓→isformed by a centre panel and two antenna side wings. In deployedconfiguration the
˓→antenna has an overall aperture of 12.3 x 0.84 m.The antenna provides a fast
˓→electronic scanning capability inazimuth and elevation and is based on low loss and
˓→Subsystem (SES) is developed and build byAstrium Ltd. It provides all radar control,
˓→ IF/ RF signalgeneration and receive data handling functions for the SARInstrument.
˓→The fully redundant SES is based on a channelisedarchitecture with one transmit and
˓→two receive chains, providing amodular approach to the generation and reception of
˓→minimisesdownlink times.',
˓→sentinel-1</a>',
'id': '04548172-c64a-418f-8e83-7a4d148adf1e',
'md5': 'E5855D1C974171D33EE4BC08B9D221AE',
'size': 4633501134,
'title': 'S1A_IW_SLC__1SDV_20170425T155612_20170425T155639_016302_01AF91_46FF',
'url': "https://scihub.copernicus.eu/apihub/odata/v1/Products('04548172-c64a-418f-
˓→8e83-7a4d148adf1e')/$value"}
1.3.4 Logging
import logging
logging.basicConfig(format='%(message)s', level='INFO')
import logging
logger = logging.getLogger('sentinelsat')
logger.setLevel('INFO')
h = logging.StreamHandler()
h.setLevel('INFO')
fmt = logging.Formatter('%(message)s')
h.setFormatter(fmt)
logger.addHandler(h)
10 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
To search for recent Sentinel 2 imagery by MGRS tile, use the tileid parameter:
query_kwargs = {
'platformname': 'Sentinel-2',
'producttype': 'S2MSI1C',
'date': ('NOW-14DAYS', 'NOW')}
products = OrderedDict()
for tile in tiles:
kw = query_kwargs.copy()
kw['tileid'] = tile # products after 2017-03-31
pp = api.query(**kw)
products.update(pp)
api.download_all(products)
NB: The tileid parameter only works for products from April 2017 onward due to missing metadata in SciHub’s DHuS
catalogue. Before that, but only from December 2016 onward (i.e. for single-tile products), you can use a filename
pattern instead:
1.3.6 API
Methods
check_files([paths, ids, directory, delete]) Verify the integrity of product files on disk.
check_query_length(query) Determine whether a query to the OpenSearch API
is too long.
count([area, date, raw, area_relation]) Get the number of products matching a query.
download(id[, directory_path, checksum]) Download a product.
download_all(products[, directory_path, . . . ]) Download a list of products.
format_query([area, date, raw, area_relation]) Create OpenSearch API query string
get_product_odata(id[, full]) Access OData API to get info about a product.
get_products_size(products) Return the total file size in GB of all products in the
OpenSearch response
query([area, date, raw, area_relation, . . . ]) Query the OpenSearch API with the coordinates of
an area, a date interval and any other search key-
words accepted by the API.
query_raw(query[, order_by, limit, offset]) Do a full-text query on the OpenSearch API using
the format specified in https://scihub.copernicus.eu/
twiki/do/view/SciHubUserGuide/3FullTextSearch
to_dataframe(products) Return the products from a query response as a Pan-
das DataFrame with the values in their appropriate
Python types.
to_geodataframe(products) Return the products from a query response as a
GeoPandas GeoDataFrame with the values in their
appropriate Python types.
to_geojson(products) Return the products from a query response as a Geo-
JSON with the values in their appropriate Python
types.
12 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
static check_query_length(query)
Determine whether a query to the OpenSearch API is too long.
The length of a query string is limited to approximately 3893 characters but any special characters (that is,
not alphanumeric or -_.~) are counted twice towards that limit.
Parameters
query [str] The query string
Returns
float Ratio of the query length to the maximum length
Notes
The query size limit arises from a limit on the length of the server’s internal query, which looks like
http://localhost:30333//solr/dhus/select?q=. . . &wt=xslt&tr=opensearch_atom.xsl&dhusLongName=Sentinels+Scientific+D
&dhusServer=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fscihub.copernicus.eu%2Fapihub%2F&originalQuery=. . .
&rows=100&start=0&sort=ingestiondate+desc
This function will estimate the length of the “q” and “originalQuery” parameters to determine whether the
query will fail. Their combined length can be at most about 7786 bytes.
count(area=None, date=None, raw=None, area_relation=’Intersects’, **keywords)
Get the number of products matching a query.
This is a significantly more efficient alternative to doing len(api.query()), which can take minutes to run
for queries matching thousands of products.
Parameters
Identical to the parameters of api.query().
Returns
int The number of products matching a query.
download(id, directory_path=’.’, checksum=True)
Download a product.
Uses the filename on the server for the downloaded file, e.g.
“S1A_EW_GRDH_1SDH_20141003T003840_20141003T003920_002658_002F54_4DD1.zip”.
Incomplete downloads are continued and complete files are skipped.
Parameters
id [string] UUID of the product, e.g. ‘a8dd0cfd-613e-45ce-868c-d79177b916ed’
directory_path [string, optional] Where the file will be downloaded
checksum [bool, optional] If True, verify the downloaded file’s integrity by checking its
MD5 checksum. Throws InvalidChecksumError if the checksum does not match. Defaults
to True.
Returns
product_info [dict] Dictionary containing the product’s info from get_product_info() as
well as the path on disk.
Raises
InvalidChecksumError If the MD5 checksum does not match the checksum on the server.
14 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
The time interval formats accepted by the ‘‘date‘‘ parameter can also be used with
any other parameters that expect time intervals (that is: ‘beginposition’, ‘endposition’,
16 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
polygon coordinates string of comma separated coordinate tuples (lon, lat) to be used by
SentinelAPI
1.3.7 Exceptions
This means that the given username/password combination is incorrect. Note that if you created your account recently
it could take a while (a few days?) until you can use that account on apihub and therefore sentinelsat too. You
can go here to test access on the apihub endpoint.
Scihub servers are known to have outages due to high demand, try again later.
Maybe there are no images for the specified time period, by default sentinelsat will query the last 24 hours only.
Anything else?
Added
Changed
Deprecated
Fixed
Added
Changed
• remove hard coded product type list from cli (#190, @lenniezelk)
• Made the function signature of count() fully compatible with query(). Irrelevant parameters are simply
ignored.
Deprecated
Fixed
Changed
• Made checksumming the default behavior, and removed its flag from the CLI. (@gbaier2)
18 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
Fixed
Added
• Option to change the type of spatial relation for the AOI in query(). The choices are ‘Interesects’, ‘Contains’
and ‘IsWithin’.
• order_by option to query() which controls the fields by which the products are sorted on the server side
before being returned. -o/--order-by on the CLI.
• limit the number of products returned by query() and to set the number of products to skip via offset.
-l/--limit on the CLI.
• Added raw parameter to query() to append any additional raw query string to the query.
• Query parameters that take intervals as values can now be passed a tuple of the interval range values.
• Date validation and parsing has been extended to all date-type parameters in queries, such as ‘ingestiondate’.
• Added count() which quickly returns the number of products matching a query on the server without retriev-
ing the full response.
• Method check_query_length to check if a query will fail because of being excessively long.
• Option to adjust the number of decimal figures in the coordinates of the WKT string returned by
geojson_to_wkt().
• CLI option to query by UUID (--uuid) or filename (--name).
• A more informative error message is shown if a too long query string was likely the cause of the query failing
on the server side. This can be useful if the WKT string length would cause the query to fail otherwise.
• Progressbars can be disabled by setting show_progressbars to False. Progressbars may be customized
by overriding the _tqdm() method.
• Contribution guidelines.
• Tests for validity of documentation and RST files.
Changed
• Merged CLI subcommands sentinel search and sentinel download into sentinelsat.
• CLI uses keywords instead of positional arguments, i.e. --user <username>.
• initial_date and end_date parameters in query() have been replaced with a single date parameter
that takes a tuple of start and end dates as input.
• Files being downloaded now include an ‘.incomplete’ suffix in their name until the download is finished.
• Removed check_existing option from download() and download_all(). Similar functionality has
been provided in the new check_files() function.
• format_query_date has been changed into a public function.
Deprecated
• query_raw() has been merged with query() and is deprecated. Use query(raw=...) instead.
Fixed
• Show the correct progress value in the download progressbar when continuing from an incomplete file. (Thanks
@gbaier!)
• Added a workaround for a server-side bug when plus symbols are used in a query.
Changed
• Replace pycurl dependency with requests. This makes installation significantly easier. (#117)
• An exception is raised in download_all() if all downloads failed.
• Change ‘Sentinels Scientific Datahub’ to ‘Copernicus Open Access Hub’ (#100)
• Renamed py.test option --vcr reset_all to --vcr reset to better reflect its true behavior.
Added
• GeoJSON footprints are allowed to contain just a single geometry instead of a feature collection. Any geometry
type that has a WKT equivalent is supported (rather than only Polygons).
• get_product_odata() can be used to get the full metadata information available for a product if
full=True is set.
• Added query_raw() that takes full text search string as input and returns a parsed dictionary just like the
updated query() method.
• CLI: --sentinel=<int> option to select satellite (constellation)
Changed
• SentinelAPI, etc. can be directly imported from sentinelsat rather than sentinelsat.sentinel.
• query() changes:
– The area argument expects a WKT string as input instead of a coordinate string. (Issue #101)
20 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
– Date arguments can be disabled by setting them to None and their values are validated on the client side.
(Issue #101)
– The return value has been changed to a dict of dicts of parsed metadata values. One entry per product
with the product ID as the key.
• download_all() expects a list of product IDs as input. This is compatible with the output of query().
• get_coordinates() has been replaced with functions read_geojson() and geojson_to_wkt().
(Issue #101)
• Use more compact and descriptive error messages from the response headers, if available.
Deprecated
• CLI: --sentinel1 and --sentinel2 will be removed with the next major release
Removed
Fixed
• Fixed invalid GeoJSON output in both the CLI and API. (Issue #104)
• Fixed broken reporting of failed downloads in the CLI. (Issue #88)
• Attempting to download a product with an invalid ID no longer creates an infinite loop and a more informative
error message is displayed in the CLI.
Added
Added
• Added to_dict, to_dataframe and to_geodataframe which convert the response content to respec-
tive types. The pandas, geopandas and shapely dependencies are not installed by default.
Changed
Added
• added a changelog
Changed
Fixed
Added
Changed
Removed
Added
22 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
Changed
Fixed
Added
Changed
1.5.15 Added
Changed
Changed
Added
Added
• Sentinel-2 support
Added
Added
Added
Changed
Added
Fixed
Added
24 Chapter 1. Contents
Sentinelsat Documentation, Release 0.12.2
• first release
• genindex
• modindex
• search
26 Chapter 1. Contents
Python Module Index
s
sentinelsat, 11
27
Sentinelsat Documentation, Release 0.12.2
D
download() (sentinelsat.SentinelAPI method), 13
download_all() (sentinelsat.SentinelAPI method), 13
F
format_query() (sentinelsat.SentinelAPI static method),
14
G
geojson_to_wkt() (in module sentinelsat), 16
get_product_odata() (sentinelsat.SentinelAPI method), 14
get_products_size() (sentinelsat.SentinelAPI static
method), 14
I
InvalidChecksumError, 17
Q
query() (sentinelsat.SentinelAPI method), 14
query_raw() (sentinelsat.SentinelAPI method), 16
R
read_geojson() (in module sentinelsat), 16
S
SentinelAPI (class in sentinelsat), 11
SentinelAPIError, 17
sentinelsat (module), 11
T
to_dataframe() (sentinelsat.SentinelAPI static method),
16
29