Welcome to Defra Data Services Forum

Hi Ella,
This change is causing me a real headache. The old ASCII files were dead easy to read and required no special software. I don't (yet) know much about the GEOTIFF format, but I can't open a TIF file with a text editor or easily read the data into my own software. This move isn't helping to democratise the data. Assuming I ever manage to figure out a way of reading a GEOTIFF file (without having to resort to a third party application), does the format give me xyz point coordinate values as the good old ASCII files did or is it just some image file?

Hi – It’s not possible to open the GeoTIFF file and extract information using basic windows software. The GeoTIFF file format can be opened in QGIS – A free and open source GIS software package. From this you could easily convert back to ASCII if you have specific software you have written that you want to open it in. The GeoTIFF file can also be read using python gdal libraries to extract the XYZ position of each pixel if you are importing into your own software or convert back into ACSCII.
The GeoTIFF File provides a XYZ position for the centre point of the pixel to the exact same positional accuracy as the previous ASCII files. As mentioned in the FAQ moving to GeoTIFFS means out LIDAR data can be support more modern web GIS applications as well as being significantly smaller which helps with storage, download and transfer costs.

Hi Ella,
Thanks for your reply. I've since figured out how to import the new data format into my Mathematica notebooks. I hear what you say about using third party conversion tools such as QGIS, etc. but would prefer not to if possible as things start to get messy if flipping between several different software applications. I do think it's a shame that you've dropped the ASCII format as it was very accessible. Why not just provide the data in both formats?

Unfortunately the DSP isn’t currently able to support multiple download formats on the open data side for raster data. It would be fantastic if both formats could be offered however this would require both formats to be stored on the system and processed adding significant storage costs and resource time to enable this. As an organisation we have also moved away from using ASCII data and we no longer store newly captured LIDAR data in this format.
Many thanks
Ella

I've been using a script to convert downloaded ASC data to JPEG2000, which results in vastly smaller file-sizes than either ASC or TIFF. It could be 32bit lossless float, but for my purposes I'm using lossy normalised 16bit uint. At some point I'll do a bit more comparitive analysis of different settings (I seem to remember Pillow complained when I tried to write float data). I'm currently using images compressed with 60db SNR (maybe a bit too aggressively lossy for some applications) and they end up being <1.5mb for a 50cm (2000x2000) tile (generally much less than that: many tiles with no void areas clocking in at ~100kb) while still giving a pretty good representation of the data.

I realise that this format isn't as widely supported as others. There are open geospatial consortium standards for it though, and it should be ok in GIS software. So anyway, it seems to me that if more people got used to using it, there may be great benefits to offering the data in that format. I'd probably stop short of recommending it for definitive archival purposes, though. If there were to be a second choice available, I'd like it to be that. I really want to get hold of the complete set of survey in that format and would be happy to share scripts etc if that helps (I've been meaning to make a request for the data in its existing form, but thought I'd I'll mention it now since I'm here).

FWIW it looks as though Mathematica also supports it, although the devil is in the details and I'm pretty sure it should support TIFF as well, but vagueries of precise versions can be problematic and I doubt it'd be any easier for Ian (who I'm afraid is probably going to have to learn to adapt). I must concur that I often find TIFF pretty irritating to work with in various ways.

Hello Ella,
My GIS software supports GeoTIFF but when I download lidar data from the DEFRA website and open the .zip I get 8 files none of which are .geoTIFF or anything like it. In case it's not obvious, I know virtually nothing about the workings of GIS and am currently at a loss to know how else to get the data that used to work so easily with .asc files. Can you help please?

Hello Ella
Problem sorted no need to respond

Thats great, thanks for letting me know

Michael. I needed to know the answer to that question as I haven't worked it out.
Hi Ella, How do I use the 8 files in the .zip folder, please?
Do you know how I can use the LIDAR data in a HEC-RAS software to analyse river flood problems, please?
Thanks.
Richard

Hi Richard,
These files make up the shapefile metadata index catalogue for the downloaded 5km GeoTIFF surface model which tells you information about when the data was flown etc. A shapefile is made up of the various components, hence why there are 8 files. When you open the shapefile in a GIS it will likely only show a single file (usually the .shp) which you add to your map.

The actual surface model GeoTIFF is in the root folder of the zip file. The GeoTIFF is made up of 3 files:
- *.tif – the surface model
- *.tfw – the world file telling it where it resides
- *.tif.xml – xml metadata which when opened in supported software provides metadata information about the GeoTIFF such as when the data was flown. This information is also in the index shapefile.

I’m not able to help with the software question as I’ve not heard of it I’m afraid – can you go to your suppliers and ask for advice.

I hope this helps

Hi,
Are DSM Composite data also available in GeoTIFF format or is only DTM currently accessible?
Many thanks,
Rose

Hi Rose,
Currently on the DSM Composite is only available for the 2017 version in ASCII format, not GeoTIFF,
We will soon be releasing the 2020 DTM composite, followed by the 2020 DSM composite at 1m and 2m resolutions (mid-summer). At which point we will retire the 2019 and 2017 versions.

I hope this helps
Data Sharing & Access team

Hi

I'm trying to download a large area of DTM - LIDAR Composite DTM 2020 - 1m or the data from the National Lidar Program. I have a shapefile of our area, however as it is a large area I always get an error saying the area is too large. Is there anyway of downloading large areas? (Full National park plus buffer.) Usually we have to get the data sent on a memory stick. Thanks

Sandra

Hi Sandra,

Please see https://support.environment.data.gov.uk/hc/en-gb/articles/360009247332-Can-I-download-large-areas-of-LIDAR-data-

Regards,
Dan, Data Sharing & Access Team

Hi Dan, thank you. I did do that actually, but it seems to have gone in as a FOI, which I seem to get various emails about, I did this on 20 July, but still waiting for the data. It isn't really an FOI as the data is freely available. Maybe I will put the request on again.

thanks

Sandra

Sandra,

Let me look into it for you. Don't raise another request yet.

Regards,
Dan, Data Sharing & Access Team

Hi Ella,

I have imported the tiff files in Civil 3D, but the contours does not overlay above the location for which I have downloaded them, can you please guide me with the coordinate system which I should use to get the contours at the right location

Sandra, when I have large areas to download with data transmission limits then I download OS Tile by OS Tile and stitch them all together in the relevant software I am using. For me this is related to large river catchments for hydrology.
Abhiyantesh, you have to make sure your GeoTIFFs are located correctly in relation to the OS map grid (assuming this is for UK). Your receiving software has to know the correct map projection so it 'straightens out' the curved surface of the earth into a planar engineering layout.
Regards, Richard Newell (Network Rail)

Thanks Rischard, I am using OSGB 1936 British National Grid coordinate system, however, still the contours are landing quite far away from the stretch that I need it for.

Good morning Abhiyantesh, is the translation error consistent? Are you using a local grid with a constant that has slipped in? I have never had that problem to be honest. Regards, Richard

Afternoon to you all, what we would like is to have GeoTIFFs of the latest DSM for the area of Exmoor National Park, so I wondered what would be the latest year. I see that there have been numerous years flown, but it seems that only the DTM's are available. If anyone could point me in the right direction, we would like do a download and use it with other height data. Many thanks Matt

Hi, You can check availability of data using the Coverage metadata files - Web Map Service preview option (link to it is available in each of the metadata records for the LIDAR products we publish). Here it is for ease: https://environment.data.gov.uk/DefraDataDownload/?Mode=OGCPreview&mapService=https%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fsurvey-index-files%2Fwms

We have just published the 2022 LIDAR Composite DSM datasets in both 1 and 2 meter resolutions (alongside of other 2022 Lidar products), I hope you will find data you require there.

Please let us know if you need any further help with this. Thanks, Joanna