Show append support in list_drivers

[martinfleis]

Fiona shows which drivers support appending in fiona.supported_drivers. Is there a way for pyogrio to do the same?

In [1]: import fiona

In [2]: fiona.supported_drivers
Out[4]: 
{'DXF': 'rw',
 'CSV': 'raw',
 'OpenFileGDB': 'raw',
 'ESRIJSON': 'r',
 'ESRI Shapefile': 'raw',
 'FlatGeobuf': 'raw',
 'GeoJSON': 'raw',
 'GeoJSONSeq': 'raw',
 'GPKG': 'raw',
 'GML': 'rw',
 'OGR_GMT': 'rw',
 'GPX': 'rw',
 'Idrisi': 'r',
 'MapInfo File': 'raw',
 'DGN': 'raw',
 'PCIDSK': 'raw',
 'OGR_PDS': 'r',
 'S57': 'r',
 'SQLite': 'raw',
 'TopoJSON': 'r'}

[jorisvandenbossche]

I am not directly sure if GDAL provides a way to query that information. In Fiona, those values are hardcoded in their drvsupport.py

In pyogrio, for listing the drivers that support writing, we use the DCAP_CREATE capability:

pyogrio/pyogrio/_ogr.pyx

Lines 103 to 129 in dff672c

	def ogr_driver_supports_write(driver):
	# check metadata for driver to see if it supports write
	if _get_driver_metadata_item(driver, "DCAP_CREATE") == 'YES':
	return True
	return False
	def ogr_list_drivers():
	cdef OGRSFDriverH driver = NULL
	cdef int i
	cdef char *name_c
	drivers = dict()
	for i in range(OGRGetDriverCount()):
	driver = OGRGetDriver(i)
	name_c = <char *>OGR_Dr_GetName(driver)
	name = get_string(name_c)
	if ogr_driver_supports_write(name):
	drivers[name] = "rw"
	else:
	drivers[name] = "r"
	return drivers

But that doesn't say anything about append support.

Looking at the code of ogr2ogr, it seems they first try to open in append mode, and if that fails open in normal mode, and if that works raise a specific error message about not being able to open an existing data source.
So they also only check that while running the write operation, and don't query some information somewhere.

This led me to realize that our current error message for failing append is not really great. We have this part updating the error message from GDAL:

pyogrio/pyogrio/_io.pyx

Lines 165 to 171 in dff672c

	except CPLE_BaseError as exc:
	if str(exc).endswith("a supported file format."):
	raise DataSourceError(
	f"{str(exc)} It might help to specify the correct driver explicitly by "
	"prefixing the file path with '<DRIVER>:', e.g. 'CSV:path'."
	) from None
	raise DataSourceError(str(exc)) from None

But that message is not very helpful when the reason for this failure is that append is not supported.

In [4]: df = geopandas.GeoDataFrame({"col": np.random.randn(10), "geometry": geopandas.points_from_xy(range(10), range(10))})

In [5]: pyogrio.write_dataframe(df, "test.gml")   # works fine

In [6]: pyogrio.write_dataframe(df, "test.gml", append=True)
...
File pyogrio/_io.pyx:169, in pyogrio._io.ogr_open()

DataSourceError: 'test.gml' not recognized as a supported file format. It might help to specify the correct driver explicitly by prefixing the file path with '<DRIVER>:', e.g. 'CSV:path'.

[theroggy]

New feature in GDAL 3.11 makes this possible: https://gdal.org/en/latest/development/rfc/rfc106_update_metadata.html

[jratike80]

Do GeoJSON and FlatGeobuf really belong to the list of "append" formats in fiona.supported_drivers? I suppose that "a" stands for append, and append means that it is possible to add data into existing datastore without re-writing all data, right? Maybe they belong, I am just wondering.