HLU Tool - Technical Guide

HLU Tool - Technical GuideRelease 3.1.3

Andy Foy

Jun 16, 2020

Contents

List of Figures 1

1 Preface 51.1 Recommended Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Reading Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Useful links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.5 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.6 Conventions used in this manual . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Installation 92.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Latest Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Installing the HLU Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Configuration 133.1 Database Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 GIS Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3 Configuring Lookup Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.4 Configuring Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4 Linking Access to SQL Server 354.1 Linking a new Access database . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2 Creating a new DSN file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.3 Updating a linked table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5 Database 515.1 Data Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2 Lookup Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635.3 Export Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685.4 Table Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

6 Database Updater 816.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 816.2 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

7 Optimisation 83

i

7.1 GIS Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 837.2 Database Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

8 Appendix 878.1 GNU Free Documentation License . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Index 89

ii

List of Figures

1 ArcMap Extensions dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1 Connection Type dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 OleDb Connection dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Data Link Properties Provider dialog . . . . . . . . . . . . . . . . . . . . . . . . . 164 Data Link Properties Connection dialog . . . . . . . . . . . . . . . . . . . . . . . . 165 SQL Server Connection dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 Select GIS Application dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Map Document selection dialog (ArcGIS) . . . . . . . . . . . . . . . . . . . . . . . 198 HLU layer selection dialog (ArcGIS) . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Save New Map Document dialog (ArcGIS) . . . . . . . . . . . . . . . . . . . . . . 2110 Format of the lut_user table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2211 Format of the lut_sources table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2412 Format of the lut_process table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2513 Format of the lut_habitat_class table . . . . . . . . . . . . . . . . . . . . . . . . . 2614 Format of the lut_habitat_type table . . . . . . . . . . . . . . . . . . . . . . . . . 2715 Format of the lut_ihs_habitat table . . . . . . . . . . . . . . . . . . . . . . . . . . 2816 Format of the lut_legacy_habitat table . . . . . . . . . . . . . . . . . . . . . . . . 2917 Format of the lut_osmm_ihs_xref table . . . . . . . . . . . . . . . . . . . . . . . . 3018 Format of the exports table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3119 Format of the exports_fields table . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1 External Data tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 Get External Data - OBDC Database dialog . . . . . . . . . . . . . . . . . . . . . . 373 Select Data Source dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Select Link Tables dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 Access Objects Navigation Pane . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 Create New Data Source - Select Driver dialog . . . . . . . . . . . . . . . . . . . . 417 Create New Data Source - Advanced DSN dialog . . . . . . . . . . . . . . . . . . . 428 Select Data Source dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 ODBC Setup Summary dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4310 Create New Data Source - Select Server dialog . . . . . . . . . . . . . . . . . . . . 4411 Create New Data Source - Authentication dialog . . . . . . . . . . . . . . . . . . . 4512 Create New Data Source - Select Database dialog . . . . . . . . . . . . . . . . . . 4513 Create New Data Source - Summary dialog . . . . . . . . . . . . . . . . . . . . . . 46

1

HLU Tool - Technical Guide, Release 3.1.3

14 External Data tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4715 Linked Table Manager dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4816 Linked Table Manager selected tables . . . . . . . . . . . . . . . . . . . . . . . . . 4817 Linked Table Manager Success dialog . . . . . . . . . . . . . . . . . . . . . . . . . 49

1 User Interface INCID Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522 User Interface Habitats Tab Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 533 User Interface Details Tab Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544 User Interface Sources Tab Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 User Interface OSMM Updates Fields . . . . . . . . . . . . . . . . . . . . . . . . . 556 User Interface History Tab Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 Database Relationships - Data Tables . . . . . . . . . . . . . . . . . . . . . . . . . 718 Database Relationships - IHS Lookup Tables . . . . . . . . . . . . . . . . . . . . . 729 Database Relationships - Priority Habitat Tables . . . . . . . . . . . . . . . . . . . 7310 Database Relationships - Habitat Tables . . . . . . . . . . . . . . . . . . . . . . . . 7411 Database Relationships - Habitat Type Tables . . . . . . . . . . . . . . . . . . . . . 7512 Database Relationships - Sources Tables . . . . . . . . . . . . . . . . . . . . . . . 7613 Database Relationships - History Tables . . . . . . . . . . . . . . . . . . . . . . . . 7714 Database Relationships - OS MasterMap Update Tables . . . . . . . . . . . . . . . 7815 Database Relationships - Other Tables . . . . . . . . . . . . . . . . . . . . . . . . 79

2 List of Figures


Permission is granted to copy, distribute and/or modify this document underthe terms of the GNU Free Documentation License, Version 1.3 or any laterversion published by the Free Software Foundation; with no InvariantSections, no Front-Cover Texts and no Back-Cover Texts. A copy of thelicense is included in the Appendix section.

List of Figures 3


4 List of Figures

CHAPTER1

Preface

The most up to date version of this documentation can be found in HTML and PDF form onReadTheDocs at readthedocs.org/projects/hlutool-technicalguide.

1.1 Recommended Knowledge

1.1.1 Administrators

We recommend that a person within each organisation should be designated as the tool anddatabase administrator. This person should:

• Have an understanding and experience of IT systems management.

• Understand relational database structures.

• Be an expert user of the chosen database system.

• Have qualifications, certified training or equivalent experience in managing databasesusing that system.

• Have certified training or equivalent experience in advanced features of the relevant GISsoftware, including defining, joining and exporting layers and creating spatial and at-tribute indexes.

1.1.2 Users

A user guide is also available on ReadTheDocs at readthedocs.org/projects/hlutool-userguidefor those who will be regular users of the HLU Tool but are not concerned with how to installor configure the tool or how to perform database administration.

5

https://readthedocs.org/projects/hlutool-technicalguide/

https://readthedocs.org/projects/hlutool-userguide/


1.2 Reading Guide

This Preface explains a little about the HLU Tool, the community of people who develop anduse it, and the licensing conditions for using and distributing it and the associated guides.

Installation (page 9) details the system requirements and describes how to install the HLU Tool.

Configuration (page 13) describes how to connect to a HLU Tool database and GIS application.

Linking Access to SQL Server (page 35) provides instructions on how to link an Access front-endto a SQL Server database.

Database (page 51) outlines the database structure and how to adapt the data for local require-ments.

Database Updater (page 81) describes the mechanism for apply updates to the database struc-ture and contents.

Optimisation (page 83) introduces tips for optimising the performance of the tool.

Appendix (page 87) contains a copy of the GNU Free Documentation License v1.3 covering thisguide.

1.3 Licensing

The code for the HLU Tool is ‘open source’ and is released under GNU General Public License(GPL) v3. Users are free to install it on as many computers as they like, and to redistribute itaccording to the GPLv3 license.

This guide is released under GNU Free Documentation License (FDL) v1.3. Permission isgranted to copy, distribute and/or modify this document under the terms of the license.

Please remember, however, that the HLU Tool cost a lot of money to develop and stillrequires further development and ongoing support. Hence any contributions towardscosts would be gratefully received. Enquiries can be made via the Knowledge Hub atkhub.net/group/association-of-local-environmental-records-centres.

1.4 Useful links

Related community links:

Users For announcements, bug reports, user Q&A and feature discussions seekhub.net/group/association-of-local-environmental-records-centres/group-forum

Administrators For release notes and installers for ArcGIS and MapInfo systems seegithub.com/HabitatFramework/HLUTool/releases

Developers For the source code for the HLU Tool see github.com/HabitatFramework/HLUTool

Issues For details of any known issues and proposed change requests seegithub.com/HabitatFramework/HLUTool/issues

6 Chapter 1. Preface

http://www.gnu.org/licenses/gpl.html

http://www.gnu.org/licenses/gpl.html

http://www.gnu.org/licenses/fdl.html

https://khub.net/group/association-of-local-environmental-records-centres

https://khub.net/group/association-of-local-environmental-records-centres/group-forum

https://github.com/HabitatFramework/HLUTool/releases

https://github.com/HabitatFramework/HLUTool

https://github.com/HabitatFramework/HLUTool/issues


1.5 Acknowledgements

Many thanks are due to all the LERCs in the south-east of England and their staff who have, andcontinue to, fund and contribute to the HLU Tool. It takes a small army of developers, testersand users to build a truly useful tool.

1.5. Acknowledgements 7


1.6 Conventions used in this manual

The following typographical conventions are used in this manual:

Ctrl-A Indicates a key, or combination of keys, to press.

Commit Indicates a label, button or anything that appears in user interfaces.

Tools. . . –> About Indicates a menu choice, or a combination of menu choices, tab selectionsor GUI buttons.

C:\Program Files\HLU Tool Indicates a filename or directory name.

Tip: Tips can help save time or provide shortcuts.

Note: Notes explain things in more detail or highlight important points.

Caution: Warnings where administrators should pay attention.

8 Chapter 1. Preface

CHAPTER2

Installation

2.1 System Requirements

2.1.1 Hardware

System Requirements

Hard disk space requirements are given as a guideline. The actual amount of disk spacerequired will depend upon the size of the GIS layers and database files. In addition to thesefiles space is required for temporary files during processing.

Minimum specification:

• 3 GHz Pentium PC

• 4 Gb RAM

• 3 Gb available hard disk space

Recommended specification:

• 3 GHz Dual Core PC

• 8 Gb RAM

• 20 Gb available hard disk space

Tip: For increased performance a multiple core PC with as much RAM as possible is recom-mended.

9


2.1.2 Software

Minimum software specification:

• .NET framework 3.5 Service Pack 1

• Microsoft Access 2000

• Microsoft SQL Server 2008 Express

• ArcGIS 10.1 or MapInfo 8.0

Recommended software specification:

• .NET framework 4.5 or above

• Microsoft Access 2016 or later

• Microsoft SQL Server 2016 Express or Microsoft SQL Server 2016 or later

• ArcGIS 10.1 and later or MapInfo 8.0 and later (32bit versions only)

2.2 Latest Release

The latest release of the tool can be downloaded from GitHub. Two versions of the tool areavailable:

• For ArcGIS or ArcGIS/MapInfo systems (e.g. v3.1.3). Use when ArcGIS is installed orwhen ArcGIS and MapInfo are both installed.

• For MapInfo only systems (e.g. v3.1.3m). Use when only MapInfo is installed.

10 Chapter 2. Installation

https://github.com/HabitatFramework/HLUTool/releases


2.3 Installing the HLU Tool

To install the HLU Tool double-click on the Setup.exe and follow the instructions. By defaultthis will install the HLU Tool into the C:\Program Files\HLU\HLU GIS Tool\ directory and willoptionally display the ReadMe file. The installer will also create two shortcuts in the start menu:

• HLU Tool - Launch - this will start the tool

• HLU Tool - Reconfigure - this will start the tool using the /c switch parameter. SeeConfiguration (page 13) for details of how to connect to a HLU Tool database and GISapplication.

2.3.1 Enable HLU Extension

Important: For ArcGIS users, the tool must be enabled in ArcGIS prior to use. Open ArcMapand go to the Tools menu and select Extensions. . . .

Fig. 1: ArcMap Extensions dialog

Tick the box next to ‘HLU ArcMap Extension’ as shown in the figure ArcMap Extensions dialog(page 11) then click Close and exit ArcMap.

2.3. Installing the HLU Tool 11


12 Chapter 2. Installation

CHAPTER3

Configuration

The first time the GIS tool is used you will need to configure the database and GIS connectiondetails. Configuration settings are user specific so every user will have to set the database andGIS connection details before they can use the tool.

The connection details, together will all the other user options configurable through the tool’s‘Options’ user interface, are stored in a user.config file within the user’s windows profile.

In Windows 10 the user.config file can be found in:

C:\Users\<username>\AppData\Roaming\HLU\<appname>_<eid>_<hash>\<ver>

For example:

C:\Users\Andy\AppData\Roaming\HLU\HluGisTool.exe_Url_lm5wjls4ajbe40rbxsgduz3nv1mqy1n0\3.1.3.0

3.1 Database Connection

In principle the HLU Tool will support connections with multiple relational database systemsincluding SQL Server, Oracle, PostgreSQL and Microsoft Access, although the connections withPostgreSQL and Oracle have not been tested.

It is recommended that the database system used is SQL Server because it provides performance,reliability and volume benefits over Microsoft Access. Because PostgreSQL and Oracle use aclient/server, as does SQL Server, they will provide similar benefits over Microsoft Access butthis has not been tested with the HLU Tool.

To configure the database connection:

1. Double-click on the link in the start menu or the desktop icon (HLU GIS Tool -Launch) to launch the application.

2. Once the tool is running a ‘Connection Type’ dialog will be displayed as shownin the figure Connection Type dialog (page 14).

13


Fig. 1: Connection Type dialog

3. Select the appropriate connection type from the drop-down list and click OK.For each dialog fields highlighted in red must be completed.

Tip:

• For SQL Server databases select SQLServer.

• For Access databases select OleDb.

14 Chapter 3. Configuration


3.1.1 Connecting to Microsoft Access

To connect the HLU Tool to a Microsoft Access database containing the HLU data:

1. Launch the HLU Tool and select OleDb from the drop-down list, then clickOK. A dialog will be displayed as shown in the figure OleDb Connection dialog(page 15).

Fig. 2: OleDb Connection dialog

2. If you have an existing Microsoft Data Link (UDL) file, click UDL File. . . andbrowse to the appropriate directory.

3. Otherwise, click New to define a new connection. The Provider tab of the‘Data link properties’ dialog will be displayed as shown in the figure Data LinkProperties Provider dialog (page 16).

4. Select the relevant OLE DB Provider.

Tip:

• For .MDB databases choose ‘Microsoft Jet 4.0 OLE DB Provider’.

• For .ACCDB databases choose ‘Microsoft Office 12.0 AccessDatabase Engine OLE DB Provider’.

5. Click Next >> to display the ‘Connection’ tab of the ‘Data Link Properties’ dia-log as shown in the figure Data Link Properties Connection dialog (page 16).

6. Click to browse to an existing database. If user names and passwords aredefined for the database, untick Blank password and enter your user nameand password.

7. Click OK, then click OK on the ‘OleDb Connection’ dialog.

3.1. Database Connection 15


Fig. 3: Data Link Properties Provider dialog

Fig. 4: Data Link Properties Connection dialog



3.1.2 Connecting to SQL Server

To connect the HLU Tool to a Microsoft SQL Server containing the HLU data:

1. First attach the database using SQL Server Management Studio and configureuser permissions.

2. Launch the HLU Tool and select SQLServer from the drop-down list, then clickOK.

3. Select the correct instance of SQL Server from the drop-down list as shown inthe figure SQL Server Connection dialog (page 17).

Fig. 5: SQL Server Connection dialog

Tip: If the computer or server is listed but no services are listed e.g. P3000CA, youcan either:

• Select the computer or server, then press End or use the Arrow keys to move thecursor to the end, then type the name of the database service.

• Exit the HLU Tool, then open the SQL Server Configuration Manager. Right-click on SQL Server Browser and set the start mode as Automatic or Manual.Start the service, then re launch the HLU Tool and select the database servicefrom the list.

4. Select whether you wish to use Windows or SQL Server authentication. Thiswill depend upon how your SQL Server database has been configured.

5. If you are using SQL Server authentication, enter the user name and passwordyou use to connect to SQL Server.

6. Select the HLU database from the ‘Database’ drop-down list.

7. The Default schema defaults to ‘dbo’. If you wish to use a different schema,select it from the drop-down list, then click OK.

3.1. Database Connection 17


3.1.3 Reconfiguring the Database Connection

To re-configure only the database connection the database configuration information must bereset by starting the tool with a /d switch parameter. To reset the configuration:

1. In Windows, go to Start Menu > Run. . . .

2. If you have installed the HLU Tool in the default location, at the prompt type:

"C:\Program Files\HLU\HLU GIS Tool\HluGisTool.exe" /d

Note: If you have installed the tool in another location, enter the appropriate folderpath in double quotes followed by /d.

3. This will start the tool and prompt the user to enter new database connectiondetails.

Tip: As part of the installation of the tool a shortcut to ‘Reconfigure’ the tool will automaticallybe added to the Start menu (along with a shortcut to ‘Launch’ the tool). This shortcut usesa /c switch (instead of the /d switch above) which will reset both the database and the GISconfiguration information.



3.2 GIS Connection

When the tool is first launched it will check your system to determine which GIS applicationsare installed. If both ArcGIS and MapInfo are installed on your system a dialog will be displayedfor you to select which one to use as shown in the figure Select GIS Application dialog (page 19).

Fig. 6: Select GIS Application dialog

3.2.1 Connecting to ArcGIS/MapInfo

To connect the tool to ArcGIS or MapInfo:

1. A dialog will be displayed which allows you to open an existing ArcGIS mapdocument or MapInfo workspace which contains the HLU layer as shown in thefigure Map Document selection dialog (ArcGIS) (page 19).

Fig. 7: Map Document selection dialog (ArcGIS)

Note: For ArcGIS users this dialog may not be visible - hold Alt and press Tab on

3.2. GIS Connection 19


your keyboard to switch windows.

2. If you have an existing map document or workspace which contains the HLUlayer, browse to the file and click Open. The HLU Tool will then be ready foruse.

3. If you do not have an existing map document or workspace, click Cancel. Adialog will then be displayed as shown in the figure HLU layer selection dialog(ArcGIS) (page 20) which allows you to select the HLU layer.

Fig. 8: HLU layer selection dialog (ArcGIS)

4. Browse to the HLU layer file and click Open. You will then be prompted toselect a location to save as a new map document or workspace as shown in thefigure Save New Map Document dialog (ArcGIS) (page 21).

5. Browse to the location, enter a name for the map document or workspace andclick OK. The HLU Tool will then be ready for use.

Tip: The GIS connection details can also be reconfigured if necessary through the Tools menu.



Fig. 9: Save New Map Document dialog (ArcGIS)

3.2.2 Reconfiguring the GIS Connection

To re-configure only the GIS connection, the GIS configuration information must be reset bystarting the tool with a /g switch parameter. To reset the configuration:

1. In Windows, go to Start Menu > Run. . . .

2. If you have installed the HLU Tool in the default location, at the prompt type:

"C:\Program Files\HLU\HLU GIS Tool\HluGisTool.exe" /g

Note: If you have installed the tool in another location, enter the appropriate folderpath in double quotes followed by /g.

3. This will start the tool and prompt the user to enter new GIS connection details.

Tip: As part of the installation of the tool a shortcut to ‘Reconfigure’ the tool will automaticallybe added to the Start menu (along with a shortcut to ‘Launch’ the tool). This shortcut usesa /c switch (instead of the /g switch above) which will reset both the database and the GISconfiguration information.

3.2. GIS Connection 21


3.3 Configuring Lookup Tables

Tables in the database that are prefixed by ‘lut_’ are lookup tables and some of these can betailored to the requirements of each organisation. Examples of configuration include:

• Adding new users to enable edit capability.

• Adding new sources as reference datasets.

• Adding new legacy habitats.

• Hiding ‘non-local’ habitats, habitat classes and habitat types.

• Changing the order that the values appear in drop-down lists.

Note: Changes to the lookup tables won’t take effect for HLU Tool instances that are running.The HLU Tool will need to be closed and re-started before any lookup table changes take effect.

See also:

See Lookup Tables (page 63) for more information on lookup tables.

3.3.1 Configuring Users

New users of the HLU Tool must be added to the ‘lut_user’ table if they wish to apply anyupdates. The format of the table is shown in the figure Format of the lut_user table (page 22).

Fig. 10: Format of the lut_user table

Note:

• Users will be able to use the tool even if their user details have not been entered into thelut_user table. However, ‘[Read Only]’ will appear in the user interface title bar and theywill not be able to apply any changes.

• Users must also have edit access to the database and GIS feature layers in order to applychanges using the tool.

• Existing user records cannot be removed from the ‘lut_user’ table if they are referenced byany of the data records (i.e. if they have applied any changes to the data). This is becausedata integrity must be retained.



Caution: Bulk update permission should only be assigned to expert users and should onlybe used with caution as mistakes can have major affects on the data.

3.3. Configuring Lookup Tables 23


3.3.2 Configuring Sources

Additional sources can be added to the ‘lut_sources’ table . The format of the table is shown inthe figure Format of the lut_sources table (page 24).

Fig. 11: Format of the lut_sources table

Note: Existing source records cannot be removed from the ‘lut_sources’ table if they are refer-enced by any of the data records (i.e. if they have been used in any incid data records). This isbecause data integrity must be retained.



3.3.3 Configuring Processes

New processes can be added to the ‘lut_process’ table. The format of the table is shown in thefigure Format of the lut_process table (page 25).

Fig. 12: Format of the lut_process table



3.3.4 Configuring Habitat Classes

Habitat Classes can be flagged as local or not in the ‘lut_habitat_class‘ table. The format of thetable is shown in the figure Format of the lut_habitat_class table (page 26).

Fig. 13: Format of the lut_habitat_class table

Setting the local flag of a Habitat Class to ‘False’ (zero) in the ‘lut_habitat_class’ table will stopit appearing in the ‘Habitat Class’ drop-down list in the Habitats tab of the main window and inthe ‘Habitat Class’ drop-down list in the Sources tab. This enables entire habitat classes that arenot used by the organisation to be hidden to keep the drop-down list as short as possible.

Note: Only Habitat Classes that are indirectly referenced by records in the‘lut_habitat_type_ihs_habitat’ translation table (by referencing one or more code_habitat_typevalues that relate to a habitat_class_code in the ‘lut_habitat_types’ table and subsequently aHabitat Class code in the ‘lut_habitat_class’ table) will appear in the ‘Habitat Class’ drop-downlist in the Habitats tab of the main window, even if the is_local flag in the ‘lut_habitat_type’ ta-ble is set to ‘True’ (minus one). This is because the ‘lut_habitat_type_ihs_habitat’ table controlsthe cross-reference between different Habitat Classes/Types and IHS Habitats.



3.3.5 Configuring Habitat Types

Habitat Types can be flagged as local in the ‘lut_habitat_type‘ table. The format of the table isshown in the figure Format of the lut_habitat_type table (page 27).

Fig. 14: Format of the lut_habitat_type table

Setting the local flag of a Habitat Type to ‘False’ (zero) in the ‘lut_habitat_type’ table will stopit appearing in the ‘Habitat Type’ drop-down list in the Habitats tab of the main window andin the ‘Habitat Type’ drop-down list in the Sources tab. This enables specific habitat types thatare not used by the organisation, or are not found in the local area, to be hidden to keep thedrop-down list as short as possible.

Note: Only Habitat Types that are directly referenced by records in the‘lut_habitat_type_ihs_habitat’ translation table (by referencing one or more code_habitat_typevalues that relate to a Habitat Type code in the ‘lut_habitat_type’ table) will appear inthe ‘Habitat Type’ drop-down list in the Habitats tab of the main window, even if theis_local flag in the ‘lut_habitat_type’ table is set to ‘True’ (minus one). This is becausethe ‘lut_habitat_type_ihs_habitat’ table controls the cross-reference between different HabitatClasses/Types and IHS Habitats.



3.3.6 Configuring IHS Habitats

IHS Habitats can be flagged as local in the ‘lut_ihs_habitat‘ table. The format of the table isshown in the figure Format of the lut_ihs_habitat table (page 28) (some columns have beenhidden).

Fig. 15: Format of the lut_ihs_habitat table

Note: Only IHS Habitats flagged as local will appear in the ‘IHS Habitat’ drop-down list in themain window. This enables habitats that are not found in the local area to be hidden to avoidbeing selected in error (e.g. coastal habitats in land-locked counties.)



3.3.7 Configuring Legacy Habitats

Legacy habitats can be configured in the ‘lut_legacy_habitat‘ table. The format of the table isshown in the figure Format of the lut_legacy_habitat table (page 29).

Fig. 16: Format of the lut_legacy_habitat table

Note: Existing legacy habitat records cannot be removed from the ‘lut_legacy_habitat’ table ifthey are referenced by any of the data records (i.e. if they have been used in any incid datarecords). This is because data integrity must be retained.



3.3.8 Configuring OSMM to IHS cross-reference

The OS MasterMap to IHS cross-reference can be configured in the ‘lut_osmm_ihs_xref‘ table.The format of the table is shown in the figure Format of the lut_osmm_ihs_xref table (page 30).

Fig. 17: Format of the lut_osmm_ihs_xref table

Note: Existing OS MasterMap to IHS cross-reference records cannot be removed from the‘lut_osmm_ihs_xref’ table (as they will be referenced by one or more of the data records (i.e.if they will have been used in the incid_osmm_updates table). This is because data integritymust be retained.



3.4 Configuring Exports

Export formats must be pre-configured before they can be used in HLU Tool.

3.4.1 Adding export formats

Export formats can be added or removed in the ‘exports’ table shown in the figure Format of theexports table (page 31).

Fig. 18: Format of the exports table

Once a new export format has been added to the ‘exports’ table the fields to be included in theexport must be added to the ‘exports_fields’ table.

3.4.2 Adding fields to an export format

The ‘exports_fields’ table shown in the figure Format of the exports_fields table (page 31) defineswhich fields are exported for each export type in the ‘exports’ table.

Fig. 19: Format of the exports_fields table

Note: GIS controlled fields such as obj, shape, perimeter, area, x, y, etc. should not be included.

3.4. Configuring Exports 31


These fields will be automatically added to the exported layer.

See also:

See Export Tables (page 68) for more information.

3.4.3 Field Formats

The format of some export fields can be modified in the output file.

Lookup related fields The format of all fields that relate to a lookup ‘lut_’ table record can bemodified using the following formats:

Table 1: Valid Export Field Formats for fields with relatedlookup tables

FieldFormat

Description Example

Code (orblank)

Outputs only the raw ‘code’ value of thespecified field.

‘GA0’.

Lookup Outputs only the ‘description’ field value fromthe relevant lookup table.

‘AcidGrassland’.

Both Outputs both the ‘code’ and ‘description’ valuesseparated by ‘ : ‘.

‘GA0 : AcidGrassland’.

Note:

• The above ‘field_format’ values (i.e. ‘Code,’ ‘Lookup’ and ‘Both’) are case sensitive.

• The ‘field_type’ must be ‘10’ (text) for the specified field.

• The ‘field_length’ must be long enough to contain the specified output format (up to 254chars) or it will be truncated.

Source date fields The format of the ‘source_date_start’ and ‘source_date_end’ fields in the‘incid_sources’ table can be modified using the following field formats:



Table 2: Valid Export Field Formats for source date fieldsFieldFormat

Output Description Example

blank Start or End date in the formatentered.

‘Jul 2008’ or ‘Nov 2009’

‘v’ Both Start and End dates in theformat entered.

‘Jul 2008 - Nov 2009’.

‘dd/MM/yyyy’Start or End date as ‘day/month/year’. ‘01/07/2008’ or‘01/11/2009’.

‘mmmyyyy’

Start or End date as ‘month year’. ‘Jul 2008’ or ‘Nov 2009’.

‘yyyy’ Start or End date as ‘year’ only. ‘2008’ or ‘2009’.‘D’ Start or End date in the vague ‘day’

format.‘01/07/2008’ or‘01/11/2009’.

‘O’ Start or End date in the vague ‘monthyear’ format.

‘Jul 2008’ or ‘Nov 2009’.

‘Y’ Start or End date in the vague ‘year’format.

‘2008’ or ‘2009’.

‘P’ Start or End date in the vague ‘seasonyear’ format.

‘Summer 2008’ or‘Autumn 2009’.

Note:

• The above ‘field_format’ values are case sensitive.

• The ‘field format’ value ‘v’ can be used with wither the ‘source_date_start’ or‘source_date_end’ fields.

• The ‘field_type’ must be ‘10’ (text) for the specified field.

• The ‘field_length’ must be long enough to contain the specified output format (up to 254chars) or it will be truncated.

Caution:

• When using the field format ‘dd/MM/yyyy’ the month portion ‘MM’ must be in capi-tals (lower case ‘mm’ means ‘minutes’ not ‘Months’).

• Because of the way Source dates are stored in the database, dates entered as a sin-gle date (e.g. ‘01/07/2008’ or ‘2008’) rather than a date range (e.g. ‘01/07/2008 -30/11/2009’ or ‘- 2008’) will always have a ‘source_date_end’ of ‘Unknown’ or blank(depending on the chosen output format).

• Vague dates (e.g. ‘Jul 2008’ or ‘2008’) are stored based on the first day of the relevantperiod, so if output in a more precise format (e.g. entered as ‘2008’ but output as‘mmm yyyy’) the day and/or month output will be the first day/month of the relevantperiod.

Date field specifiers The following table describes the valid date and time format specifiers.

3.4. Configuring Exports 33


Table 3: Valid date and time format specifiersSpecifier Description“d” The day of the month, from 1 through 31.“dd” The day of the month, from 01 through 31.“ddd” The abbreviated name of the day of the week.“dddd” The full name of the day of the week.“h” The hour, using a 12-hour clock from 1 to 12.“hh” The hour, using a 12-hour clock from 01 to 12.“H” The hour, using a 24-hour clock from 0 to 23.“HH” The hour, using a 24-hour clock from 00 to 23.“m” The minute, from 0 through 59.“mm” The minute, from 00 through 59.“M” The month, from 1 through 12.“MM” The month, from 01 through 12.“MMM” The abbreviated name of the month.“MMMM” The full name of the month.“s” The second, from 0 through 59.“ss” The second, from 00 through 59.“t” The first character of the AM/PM designator.“tt” The AM/PM designator.“y” The year, from 0 to 99.“yy” The year, from 00 to 99.“yyyy” The year as a four-digit number.“:” The time separator.“/” The date separator.space Date or time spacing character.


CHAPTER4

Linking Access to SQL Server

Using SQL Server as the relational database system for the HLU Tool will provide performance,reliability and volume benefits over Microsoft Access. But if some of the users don’t have au-thority to use SQL Server’s Management Studio (SSMS), or aren’t familiar with using SSMS,then one solution is to create an Access database front-end and link it to the HLU Tool databasein SQL Server.

Many users will be more familiar with Access, so linking Access to SQL Server in this wayallows users to view and edit the HLU Tool data without needing authority to use SSMS andwithout any risk of altering the database structure or settings. Using Access as a front-end tothe database in this way also allows users to create and share Access queries, forms and reportsspecific to their needs.

Note: Database administrators need to ensure that users of the Access database have thenecessary permissions to view and edit the data tables on the target SQL Server database.

35


4.1 Linking a new Access database

The process of linking an Access database to a SQL Server database can be complex and neces-sitates that you know the connection settings for the target database. The following instructionsdescribe how to link to an existing installation of the HLU Tool database running on SQL Server.It should not matter which version (i.e. SQL Server or SQL Server Express) is installed or whichrelease, as long as it is meets the System Requirements (page 9).

Note: It is also possible to link the exporter database to other relational databases, such asPostGIS, but instructions for this are not provided in this guide.

Other Access versions

The steps in this procedure are based on using Microsoft Access 2012. Instructions for differ-ent versions of Microsoft Access may vary.

The following instructions describe how to link an Access database to a SQL Server database:

1. Make sure your SQL Server service is running and the HLU Tool database isattached.

2. Open the destination Access database in which you wish to create the linkedtables.

3. On the ‘External Data’ tab, in the ‘Import & Link’ group, click ODBC Databaseas shown in External Data tab (page 36).

Fig. 1: External Data tab

36 Chapter 4. Linking Access to SQL Server


4. Select Link to the data source by creating a linked table (as shown in Get ExternalData - OBDC Database dialog (page 37)) and then click OK.

Fig. 2: Get External Data - OBDC Database dialog

4.1. Linking a new Access database 37


5. In the ‘Select Data Source’ dialog box as shown in Select Data Source dialog(page 38), if the DSN file you want to use already exists, locate and select theDSN file. If you haven’t yet created a DSN file for the target database skip toCreating a new DSN file (page 41) then continue from the next step.

Fig. 3: Select Data Source dialog

Note: If have haven’t already created a data source name (DSN) file for the HLUTool database see Creating a new DSN file (page 41).



6. Click OK. Access will display the ‘Link Tables’ dialog box as shown in Select LinkTables dialog (page 39).

Fig. 4: Select Link Tables dialog

7. Under ‘Tables’, click each table that you want to link to, and then click OK.

Note: Many of the tables in the list are internal SQL Server tables. Do not selectthese - only select the HLU Tool export, data and lookup tables.

8. If the ‘Select Unique Record Identifier’ dialog box appears, Access was unableto determine which field or fields uniquely identify each row of the source data.In this case, select the field or combination of fields that is unique for each row,and then click OK. If you are not sure, check with the SQL Server databaseadministrator.

4.1. Linking a new Access database 39


9. If the link is successful Access will display the new linked tables in the ObjectsNavigation Pane as shown in Access Objects Navigation Pane (page 40).

Fig. 5: Access Objects Navigation Pane



4.2 Creating a new DSN file



The following instructions describe how to create a new .dsn file for the HLU Tool database:

1. Click New to create a new data source name (DSN) file. The ‘Create New DataSource’ wizard will start.

2. Select SQL Server in the list of drivers (as shown in Create New Data Source -Select Driver dialog (page 41)) and then click Next. If you are connecting to adifferent database then select the relevant driver.

Fig. 6: Create New Data Source - Select Driver dialog

4.2. Creating a new DSN file 41


3. If you wish to enter the ‘Server Name’ and ‘Database Name’ at this stage clickon Advanced. . . and enter them under the DRIVER keyword (see Create NewData Source - Advanced DSN dialog (page 42) for example). Then click OK toreturn to the ‘Create New Data Source’ wizard.

Fig. 7: Create New Data Source - Advanced DSN dialog

Note: If you don’t enter the ‘Server Name’ and ‘Database Name’ hereyou will be prompted for them later.

4. Click Next and then choose a suitable file path and file name for your new DSN.Then click Save.



5. The file path and file name of the select DSN will be displayed (see Select DataSource dialog (page 43) for example). Click Next.

Fig. 8: Select Data Source dialog

6. A summary of the DSN settings will be shown (see ODBC Setup Summary dialog(page 43) for example). Click Finish.

Fig. 9: ODBC Setup Summary dialog



7. If you didn’t enter the server and database names earlier you will be promptedfor them now. Enter a description for the data source and then from the listchoose which server you want to connect to (see Create New Data Source - SelectServer dialog (page 44) for example). Then click Next.

Fig. 10: Create New Data Source - Select Server dialog

Tip: If the server doesn’t appear in the list then manually type theserver name it into the Server field



8. Choose either Windows NT authentication or SQL Server authentication, asshown in see Create New Data Source - Authentication dialog (page 45), depend-ing on how the security settings have been defined in the SQL Server databasethen click Next. If you are not sure, check with the SQL Server database ad-ministrator.

Fig. 11: Create New Data Source - Authentication dialog

9. Select the ‘Change the default database to’ checkbox and then from the list se-lect which database to connect to (see Create New Data Source - Select Databasedialog (page 45) for example). Then click Next.

Fig. 12: Create New Data Source - Select Database dialog

10. Leave all the settings as the default values and click Finish.



11. A summary of the ODBC Setup will be displayed (see Create New Data Source -Summary dialog (page 46) for example). Click Test Data Source . . . to ensurethe settings are correct and the connection works and then click OK to closethe test window.

Fig. 13: Create New Data Source - Summary dialog

12. Click OK to save the DSN. You will then be returned to the ‘Select Data Source’window to continue linking a new database (see Linking a new Access database(page 36)).

Note: Once you have defined a DSN for your HLU Tool database you won’t need to repeatsteps 6 to 14 if you need to link another Access database in future.



4.3 Updating a linked table

Each time you open a linked table you will see the latest data displayed in it. However, structuralchanges made to a SQL Server table are not automatically reflected in a linked table. In thiscase you will need to update the linked table by applying the latest SQL Server object structure.



The following instructions describe how to update the links from an Access database to a SQLServer database:

1. Make sure your SQL Server service is running and the HLU Tool database isattached.

2. Open the destination Access database in which you wish to create the linkedtables.

3. On the ‘Database Tools’ menu click Linked Table Manager as shown in ExternalData tab (page 47).

Fig. 14: External Data tab

4. The ‘Linked Table Manager’ dialog will appear as shown in Linked Table Man-ager dialog (page 48).

4.3. Updating a linked table 47


Fig. 15: Linked Table Manager dialog

5. Select the Always prompt for new location check box.

6. Click the tables that you wish to update or click the Select All button (see LinkedTable Manager selected tables (page 48) for example) and then click OK.

Fig. 16: Linked Table Manager selected tables

7. If the update if successful Access will display a message to that effect as shownin Linked Table Manager Success dialog (page 49). Otherwise Access will displayan error message.

8. Click OK to close the Linked Table Manager.



Fig. 17: Linked Table Manager Success dialog

4.3. Updating a linked table 49

CHAPTER5

Database

The database consists of 3 types of tables as follows:

1. Data tables

2. Lookup tables

3. Export tables

5.1 Data Tables

Tables in the database prefixed by ‘incid’ are data tables and hence contain all the attribute datarelating to the GIS features. The attributes have been separated into 8 tables to ‘normalise’ thedata which reduces storage space, improves performance and provides greater flexibility.

Key to Data Tables

1. incid

2. incid_ihs_matrix

3. incid_ihs_formation

4. incid_ihs_management

5. incid_ihs_complex

6. incid_bap

7. incid_sources

8. incid_osmm_updates

9. history

10. incid_mm_polygons

51


How the data tables relate to the fields in the user interface is demonstrated in the followingfigures:

Fig. 1: User Interface INCID Fields

52 Chapter 5. Database


Fig. 2: User Interface Habitats Tab Fields

5.1. Data Tables 53


Fig. 3: User Interface Details Tab Fields



Fig. 4: User Interface Sources Tab Fields

Fig. 5: User Interface OSMM Updates Fields

5.1. Data Tables 55


Fig. 6: User Interface History Tab Fields



5.1.1 incid

This is the main data table with one record per INCID. All the other data tables relate to thistable via the INCID field.

incid Char(12) - A unique Incremental identifier for each logical group of features.

legacy_habitat Char(50) - Foreign key to code in the ‘lut_legacy_habitat’ table rep-resenting the legacy Habitat for the INCID.

site_ref Char(16) - A free-text field containing a reference for the location of thefeature.

site_name Char(100) - A free-text field containing a name for the location of thefeature.

boundary_base_map Char(2) - Foreign key to Code column in the‘lut_boundary_map’ table representing the data map used to identify thefeature boundary.

digitisation_base_map Char(2) - Foreign key to Code column in the‘lut_boundary_map’ table representing the data map used to digitise thefeature boundary.

ihs_version Char(20) - Foreign key to ihs_version in the ‘lut_ihs_version’ table stor-ing the active version of IHS when the INCID attributes were last updated.

ihs_habitat Char(8) - Foreign key to code in the ‘lut_ihs_habitat’ table representingthe main IHS Habitat for the INCID.

general_comments Char(254) - A free-text field containing any general commentsrelating to the INCID.

created_date DateTime - The date and time that the INCID was first created (eitherduring the initial framework conversion or following a logical split).

created_user_id Char(40) - Foreign key to user_id in the ‘lut_user’ table represent-ing the user that created the INCID.

last_modified_date DateTime - The date and time that the INCID was last modi-fied.

last_modified_user_id Char(40) - Foreign key to user_id in the ‘lut_user’ table rep-resenting the user that last modified the INCID attributes or split or merged theINCID.

5.1.2 incid_ihs_matrix

This table contains any IHS Matrix codes recorded alongside an IHS Habitat code to refine thehabitat definition for an INCID. There can be between 0 and 3 records for each INCID.

matrix_id Integer - A unique ID for each record.

incid Char(12) - Foreign key to incid in the ‘incid’ table.

matrix Char(8) - Foreign key to code in the ‘lut_ihs_matrix’ table representing anIHS Matrix type.

5.1. Data Tables 57


5.1.3 incid_ihs_formation

This table contains any IHS Formation codes recorded alongside an IHS Habitat code to refinethe habitat definition for an INCID. There can be between 0 and 2 records for each INCID.

formation_id Integer - A unique ID for each record.


formation Char(8) - Foreign key to code in the ‘lut_ihs_formation’ table represent-ing an IHS Formation type.

5.1.4 incid_ihs_management

This table contains any IHS Management codes recorded alongside an IHS Habitat code to refinethe habitat definition for an INCID. There can be between 0 and 2 records for each INCID.

management_id Integer - A unique ID for each record.


management Char(8) - Foreign key to code in the ‘lut_ihs_management’ table rep-resenting an IHS Management type.

5.1.5 incid_ihs_complex

This table contains any IHS Complex codes recorded alongside an IHS Habitat code to refinethe habitat definition for an INCID. There can be between 0 and 2 records for each INCID.

complex_id Integer - A unique ID for each record.


complex Char(8) - Foreign key to code in the ‘lut_ihs_complex’ table representingan IHS Complex type.

5.1.6 incid_bap

This table contains details of the priority habitats and potential priority habitats for an INCID.There can be between 0 and 3 records for each INCID.

bap_id Integer - A unique ID for each record.


bap_habitat Char(11) - Foreign key to code in the ‘lut_habitat_type’ table repre-senting a priority habitat (or potential priority habitat).

quality_determination Char(2) - Foreign key to code in the‘lut_bap_quality_determination’ table representing the accuracy with whichthe priority habitat has been determined.

quality_interpretation Char(2) - Foreign key to code in the‘lut_bap_quality_interpretation’ table representing how well the priorityhabitat was interpreted from the source data.



interpretation_comments Char(254) - A free-text field containing any commentsto explain the reasoning behind the priority habitat determination and inter-pretation.

5.1.7 incid_sources

This table contains details of the source datasets for an INCID. There can be between 0 and 3records for each INCID.

incid_source_id Integer - A unique ID for each record.


source_id Integer - Foreign key to source_id in the ‘lut_sources’ table representinga source dataset.

source_date_start Integer - Start date of the data range covered by the sourcedataset represented as the number of days since 01/01/1900.

source_date_end Integer - End date of the data range covered by the source datasetrepresented as the number of days since 01/01/1900.

source_date_type Char(2) - String that describes the format of the date range cov-ering the source dataset.

Table 1: Vague date typesDate Type Description ExampleD Single day date 15/10/2010DD Day-to-date date range 15/10/2010 - 18/10/2010D- Day start with no end date 15/10/2010 --D Day end with no start date - 18/10/2010O Single month date Oct 2010OO Month-to-month date range Oct 2010 - Nov 2010O- Month start with no end date Oct 2010 --O Month end with no start date - Nov 2010Y Single year date 2010YY Year-to-year date range 2010 - 2011Y- Year start with no end date 2010 --Y Year end with no start date - 2011P Single season date Autumn 2010PP Season-to-season date range Autumn 2010 - Winter 2010P- Season start with no end date Autumn 2010 --P Season end with no start date - Winter 2010U Unknown date Unknown

source_habitat_class Char(5) - Foreign key to incid in the ‘lut_habitat_class’ tablerepresenting the habitat classification of the source dataset.

source_habitat_type Char(11) - Foreign key to incid in the ‘lut_habitat_type’ tablerepresenting the habitat type of the source dataset.

source_boundary_importance Char(1) - Foreign key to code in the ‘lut_important’table representing the relative importance of the source when determining theboundary location of all the features in the INCID.

5.1. Data Tables 59


source_habitat_importance Char(1) - Foreign key to code in the ‘lut_important’table representing the relative importance of the source when determining theIHS Habitat and associated multiplex codes of the INCID.

sort_order Integer - Determines the (ascending) order the sources for each INCIDwill be displayed in the ‘Sources’ tab of the main window.

5.1.8 incid_osmm_update

This table contains details of any proposed Ordnance Survey MasterMap (OSMM) updates foran INCID. There will only be OSMM update records if the habitat framework has been externallyprocessed to integrate more recent OSMM data. Any proposed updates based on the new OSMMdata will be loaded into this table.

incid_osmm_update_id Integer - A unique ID for each proposed update.


osmm_xref_id Integer - Foreign key to osmm_xref_id in the ‘lut_osmm_ihs_xref’table representing a unique set of OS MasterMap attributes.

spatial_flag Char(1) - Indicates whether part of the new feature has been changedcompared to the original framework.

process_flag Integer - Indicates which step in the external OSMM Update processthe proposed update was determined.

change_flag Char(1) - Indicate whether the proposed habitat category is the sameas the original habitat category and whether it is a higher or lower level in thehabitat hierarchy.

status Integer - Indicates the current status of the proposed OSMM Update (pro-posed, pending, applied, ignored or rejected).

created_date DateTime - The date and time that the proposed update was first cre-ated (when the framework was externally processed to integrate more recentOSMM data).

created_user_id Char(40) - Foreign key to user_id in the ‘lut_user’ table represent-ing the user that created the proposed update.

last_modified_date DateTime - The date and time that the proposed update waslast modified.

last_modified_user_id Char(40) - Foreign key to user_id in the ‘lut_user’ table rep-resenting the user that last modified the proposed update by skipping, accept-ing, rejecting or ignoring it.

5.1.9 history

This table contains record of every change to every feature made using the HLU Tool.

history_id Integer - A unique ID for each record.




toid Char(20) - The unique Ordnance Survey topographical identifier of each fea-ture.

toid_fragment_id Char(5) - An incremental number (prefixed with zeros) used asa unique reference for each fragment of a single TOID.

modified_user_id Char(40) - Foreign key to user_id in the ‘lut_user’ table repre-senting the user that modified the feature.

modified_date DateTime - The date and time that the features was modified.

modified_process Char(3) - Foreign key to code in the lut_process table represent-ing the activity being undertaken when the feature was modified.

modified_reason Char(3) - Foreign key to code in the lut_reason table representingthe underlying explanation for the change to the feature.

modified_ihs_category Char(2) - Foreign key to code in the ‘lut_ihs_category’ tablerepresenting the first 2 characters of the IHS Habitat code prior to the changesto the feature.

modified_ihs_summary Char(50) - A concatenation of all the IHS habitat and mul-tiplex codes from the INCID for this feature prior to the changes to the feature.

modified_operation Char(3) - Foreign key to code in the lut_operation table repre-senting the operation that undertaken to cause the change to the feature.

modified_incid Char(12) - The incid prior to the changes to the feature. In theevent of a logical split or logical merge this value will be different to the current‘incid’, otherwise it will be the same as the current ‘incid’.

modified_toid_fragment_id Char(12) - The toid_fragment_id prior to the changesto the feature. In the event of a physical split or logical merge this value maybe different to the current ‘toid_fragment_id’ otherwise it will be the same asthe current ‘toid_fragment_id’.

modified_length Float - A decimal value of variable precision representing theperimeter length of the feature after the changes to the feature.

modified_area Float - A decimal value of variable precision representing the spatialarea of the feature after the changes to the feature.

5.1.10 incid_mm_polygons

This table is a local database copy of the attribute table for the GIS feature layer to improveperformance. If the GIS features are split into separate GIS layers this table contains the at-tribute records for all the layers combined. There can be any number of records for each INCID,depending upon how many TOIDs and TOID fragments are associated with the INCID.


toid Char(20) - The unique Ordnance Survey topographical identifier of each fea-ture.

toid_fragment_id Char(5) - An incremental number (prefixed with zeros) used asa unique reference for each fragment of a single TOID.

ihs_category Char(2) - Foreign key to code in the ‘lut_ihs_category’ table represent-ing the first 2 characters of the IHS Habitat code.

5.1. Data Tables 61


ihs_summary Char(50) - A concatenation of all the IHS habitat and multiplex codesfrom the INCID for this feature. This field is automatically maintained by thetool.

shape_length Float - A decimal value of variable precision representing the perime-ter length of the feature.

shape_area Float - A decimal value of variable precision representing the spatialarea of the feature.



5.2 Lookup Tables

Tables in the database prefixed by ‘lut_’ are lookup tables and are used in many drop-down listsin the user interfaces to restrict choices to only valid or appropriate values for the organisation.

Some of the lookup tables contain records and settings that are generic to all HLU Tool installa-tions and hence should be considered as ‘system’ records (indicated by the system_supplied at-tribute set to ‘True’ (minus one). These records are configured centrally and updates are appliedto HLU Tool installations using the HLUDbUpdater.exe tool (see Database Updater (page 81) formore details). The remaining lookup tables can be configured for a given HLU Tool installationto tailor them to the specific requirements of each organisation.

Note:

• Changes to the lookup tables won’t take effect for HLU Tool instances that arerunning. The HLU Tool will need to be closed and re-started before any lookuptable changes to take effect.

• Lookup table values are relevant to the whole database system and hence anychanges will affect all users of that database.

• All records in tables containing a ‘sort_order’ attribute must have a numericalvalue set or they may not appear in the relevant drop-down lists.

See also:

See Configuring Lookup Tables (page 22) for more information on configuring lookup tables.

The following lookup tables can be updated to tailor local requirements:

5.2.1 lut_users

This table contains details of all the users that have editing capability with the HLU Tool andindicates if they are also able to perform ‘bulk’ updates.

user_id The user’s Windows login ID. If the user logs in to a domain then the loginshould be entered in the format: [Domain]\[LoginID].1

user_name The name which will be displayed in the ‘By’ fields of the INCID sectionand the History tab.

bulk_update Determines whether the user has permissions to run a bulk update tochange attributes for all selected records. Ticking this checkbox gives the userpermission to run bulk updates.

sort_order Determines the order user names would be displayed in any relevantdrop-down. This field is not currently used (as there are no drop-down liststhat display users.)

Caution: Bulk update permission should only be assigned to expert users andshould only be used with caution as mistakes can have major affects on the data.

1 The ‘user_id’ of the current user is shown in the Tools. . . –> About window.

5.2. Lookup Tables 63


See also:

See Configuring Users (page 22) for more information.

5.2.2 lut_sources

This table contains details of all the source datasets that can be referenced as a ‘Source’ by anINCID.

source_id A unique ID for each source.

source_name The name which appears in the ‘Name’ drop-down list in the ‘Sources’tab.

source_date_default [Optional]. If a date is entered, the ‘Vague Date’ field in the‘Sources’ tab will be set to this value (if blank) when this source is selected. Ifthe date is left blank, the ‘Vague Date’ field will not be altered.

sort_order Determines the order source names are displayed in the ‘Name’ drop-down list in the ‘Sources’ tab.

See also:

See Configuring Sources (page 24) for more information.

5.2.3 lut_legacy_habitat

This table contains details of all the legacy habitats that can be referenced as a by an INCID.

code A unique 50 character field for each legacy habitat.

description A brief description or name that will appear in the ‘Legacy Habitat’drop-down list in the main window.

sort_order Determines the order legacy habitats are displayed in the ‘Legacy Habi-tat’ drop-down list in the main window.

See also:

See Configuring Legacy Habitats (page 29) for more information.

5.2.4 lut_process

This table contains details of all the processes that can be referenced as the activity beingundertaken when applying updates with the HLU Tool.

code A unique 3 character field for each source.

description A brief description or name that will appear in the ‘Process’ drop-downlist in the main window.

sort_order Determines the order processes are displayed in the ‘Process’ drop-downlist in the main window.



5.2.5 lut_reason

This table contains details of all the reasons that can be referenced as the underlying explanationfor applying updates with the HLU Tool.

code A unique 3 character field for each source.

description A brief description or name that will appear in the ‘Reason’ drop-downlist in the main window.

sort_order Determines the order processes are displayed in the ‘Reason’ drop-downlist in the main window.

5.2.6 lut_osmm_ihs_xref

This table contains a cross-reference between all the OS MasterMap feature types and the IHShabitat and multiplex codes. It is used when reviewing and bulk applying proposed OSMMUpdates.

osmm_xref_id A unique ID for each cross-reference. This field is referenced by theincid_osmm_update table.

make An OS MasterMap attribute. Where known it indicates whether the real-world nature of the feature is man-made, natural or both (multiple), otherwisethe value is unclassified or unknown.

desc_group An OS MasterMap attribute. The primary classification of a featureassigned to one or more of 21 groups, most of which are categories of real-world topographic objects, such as path, building or natural environment.

desc_term An OS MasterMap attribute. If present gives further classification infor-mation about a feature typically specifying the natural land cover types present.

theme An OS MasterMap attribute. The theme(s) that the feature belongs to.

feat_code An OS MasterMap attribute. A numerical feature code (a five-digit inte-ger) assigned to each feature.

ihs_habitat to ihs_complex2 The IHS habitat and multiplex codes correspondingto the unique collection of above OS MasterMap attributes.

ihs_summary The consolidated summary of the above IHS habitat and multiplexcodes.

manmade Indicates if the OS MasterMap feature is considered man-made or not.The classification ‘man-made’ may also include natural features where OSMasterMap is always considered to be accurate (such as rivers, lakes, ponds,road/rail verges, etc.)

comments Any user comments relating to the cross-referencing.

5.2.7 Local Flags & Sort Orders

Regardless of whether records in a lookup table are ‘system’ supplied records or not, many canbe configured to indicate if they are applicable to an organisation. For example, many lookuptables contain a sort_order field that will determine the order that the values appear in any



related drop-down lists. Some tables also have a is_local field that can be used to ‘hide’ valuesthat are not applicable to the local area or should not be used by the organisation.

is_local Set to ‘True’ (minus 1) to include in drop-down lists, or ‘False’ (zero) toexclude from drop-down lists.

sort_order Set to a sequential, positive numeric whole number to indicate the orderrecords should appear in drop-down lists. Alternatively all records can be setto zero to use the default sort order for that table.

Note:

• Changes to the lookup tables won’t take effect for HLU Tool instances that arerunning. The HLU Tool will need to be closed and re-started before any lookuptable changes to take effect.

• Lookup table values are relevant to the whole database system and hence anychanges will affect all users of that database.

• All records in tables containing a ‘sort_order’ attribute must have a numericalvalue set or they may not appear in the relevant drop-down lists.

The following lookup tables can be updated to tailor their is_local and/or sort_order values:

lut_ihs_habitat Contains all the IHS Habitats that can be assigned to INCIDs usingthe ‘Habitat’ field on the Habitats tab of the main window.

lut_habitat_class Contains all of the Habitat Classifications that can be assigned tosources using the ‘Habitat Class’ field on the Sources tab of the main window.

lut_habitat_type Contains all of the Habitat Types that can be assigned to sourcesusing the ‘Habitat Type’ field on the Sources tab of the main window (for theselected ‘Habitat Class’).

lut_ihs_complex Contains all the IHS Complex codes that can be assigned using the‘Complex’ fields on the Habitats tab of the main window. [sort_order only]

lut_ihs_formation Contains all the IHS Formation codes that can be assigned usingthe ‘Formation’ fields on the Habitats tab of the main window. [sort_orderonly]

lut_ihs_management Contains all the IHS Management codes that can be as-signed using the ‘Management’ fields on the Habitats tab of the main window.[sort_order only]

lut_ihs_matrix Contains all the IHS Matrix codes that can be assigned using the‘Matrix’ fields on the Habitats tab of the main window. [sort_order only]

lut_bap_quality_determination Contains the BAP determination quality types thatcan be assigned to Priority Habitats and Potential Priority Habitats on the De-tails tab of the main window. [sort_order only]

lut_bap_quality_interpretation Contains the BAP interpretation quality types thatcan be assigned to Priority Habitats and Potential Priority Habitats on the De-tails tab of the main window. [sort_order only]



lut_boundary_map Contains the list of map types that can be assigned to the‘Boundary Map’ and ‘Digitisation Map’ fields on the Details tab of the mainwindow. [sort_order only]

lut_importance Contains the difference levels of Importance that can be assignedto Sources using the ‘Boundary Imp.’ and ‘Habitat Imp.’ fields on the Sourcestab of the main window. [sort_order only]

lut_legacy_habitat Contains the list of legacy habitats that can be assigned to the‘Legacy Habitat’ field on the Habitats tab of the main window. [sort_orderonly]

lut_osmm_ihs_xref Contains a cross-reference between all the OS MasterMap fea-ture types and the IHS habitat and multiplex codes. [is_local only]

lut_process Contains details of all the processes that can be referenced in the ‘Pro-cess’ field of the main window to indicate the activity being undertaken whenusing the HLU Tool. See lut_process (page 64) for more details. [sort_orderonly]

lut_reason Contains details of all the reasons that can be referenced in the ‘Reason’field of the main window to indicate the activity being undertaken when usingthe HLU Tool. See lut_reason (page 65) for more details. [sort_order only]

lut_sources Contains details of all the source datasets that can be referenced as a‘Source’ on the Sources tab of the main window. See lut_sources (page 64) formore details. [sort_order only]

lut_user Contains details of all the users that have editing capability with the HLUTool and indicates if they are also able to perform ‘bulk’ updates. See lut_users(page 63) for more details. [sort_order only]



5.3 Export Tables

Tables in the database prefixed by ‘export’ are export tables and are used to define differentformats that can be used to export data from the HLU Tool database and GIS layers to a new‘standalone’ GIS layer.

See also:

See Configuring Exports (page 31) for more information.

5.3.1 exports

This table lists all the export ‘formats’ that can be used when exporting data.

export_id A unique identifier used to determines which fields are selected from the‘exports_fields’ table.

export_name The name which will be displayed in the ‘Export Format’ drop-downlist.

Once a new export format has been added to the ‘exports’ table the fields to be included in theexport must be added to the ‘exports_fields’ table.

5.3.2 exports_fields

Excluded Fields

• GIS controlled fields such as obj, shape, perimeter, area, x, y, etc. should not be in-cluded. These fields will be automatically added to the exported layer.

• Fields already in the active GIS layer (e.g. toid, toid_fragment_id, ihs_category,ihs_summary) are automatically included in the exported layer and hence should notbe added to the exports_fields table.

This table defines which fields are to be exported for each export format in the ‘exports’ table.It also defines what the export fields will be called, the order they will appear in the new GISlayer and the number of occurrences of each field (where fields can appear in multiple tablerecords.)

export_field_id A unique identifier for the field.

export_id The unique identifier for the export type in the ‘exports’ table (see exports(page 68)).

table_name The name of the source table in the database containing the column tobe exported.

column_name The name of the column within the source table.

column_ordinal The number of the column within the source table starting from1. The export function does not require this column to be completed.



field_name The name of the column in the exported GIS layer. The ‘field_name’must be a valid ArcGIS/MapInfo column name (i.e. containing no spaces orspecial characters.)

Caution: When exporting to an ArcGIS shapefile field names must be less than10 characters or they will be truncated or renamed by ArcGIS.

field_ordinal Sets the order of the fields in the exported GIS layer.

field_count Allows users to set the number of child records to be exported. Fieldsfrom the incid table do not support field_count values as there is only ever oneincid record for an incid.

field_type Allows users to set the data type of the field to be exported.

Table 2: Valid Export Field TypesFieldType

FieldDescription

Comment

3 Integer Standard number with no decimal places.6 Single Short number with decimal places.7 Double Long number with decimal places.8 Date/Time Date and Time stamp.

10 Text Text field up to 254 characters long.99 AutoNumber Integer field that automatically increments with

each row.

field_length Allows users to set the maximum length of text fields. Text input val-ues longer than this length will be truncated during the export without warn-ing.

Note: field_length is only used where the field_type is ‘10’ (text), otherwise itis ignored.

field_format Allows users to determine the format of the exported field. See FieldFormats (page 32) for more details on which export fields can be formatted andhow to format them.

5.3. Export Tables 69


5.4 Table Relationships

There are 52 tables in the HLU Tool relational database comprised of data tables, lookup tablesand export tables. The relationships between the tables are too numerous and complex todisplay in a single diagram so the tables and relationships have therefore been separated into 7logical groups, some of which connect and overlap with one another.

Tip: Bespoke relationship diagrams between the various HLU Tool tables can be created usingSQL Server Management Studio.



5.4.1 Data Tables

Fig. 7: Database Relationships - Data Tables

5.4. Table Relationships 71


5.4.2 IHS Lookup Tables

Fig. 8: Database Relationships - IHS Lookup Tables



5.4.3 Priority Habitat Tables

Fig. 9: Database Relationships - Priority Habitat Tables



5.4.4 Habitat Tables

Fig. 10: Database Relationships - Habitat Tables



5.4.5 Habitat Type Tables

Fig. 11: Database Relationships - Habitat Type Tables



5.4.6 Sources Tables

Fig. 12: Database Relationships - Sources Tables



5.4.7 History Tables

Fig. 13: Database Relationships - History Tables



5.4.8 OS MasterMap Update Tables

Fig. 14: Database Relationships - OS MasterMap Update Tables



5.4.9 Other Tables

Fig. 15: Database Relationships - Other Tables


CHAPTER6

Database Updater

In order to apply structural and data changes to the HLU tool database you will need to usethe HLU Tool Database Updater HLUDbUpdater.exe. The HLU Tool Database Updater providesan automated mechanism of applying changes to a target HLU Tool relational database. It willprocess one or more script files and execute all the SQL commands in the files.

Source Code The source code for this program is open source and can be downloaded from<https://github.com/HabitatFramework/HLUTool-DatabaseUpdater>.

Latest Release The latest release of this program can be downloaded from <https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/releases>.

Latest Scripts The latest scripts for this program can be downloaded from <https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/archive/scripts.zip>.

6.1 System Requirements

The HLU Tool Database Updater requires the following to operate:

Hardware

• 2 GHz or higher processor

• 2 GB RAM

• 10 MB of free hard disk space

Software

• Microsoft Windows XP/2003/2008/Vista/Windows 7 or Windows 8

• Microsoft .NET Framework 3.5 SP1, 4.0, or 4.5 installed

81

https://github.com/HabitatFramework/HLUTool-DatabaseUpdater

https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/releases

https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/releases

https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/archive/scripts.zip

https://github.com/HabitatFramework/HLUTool-DatabaseUpdater/archive/scripts.zip


6.2 Instructions

The program does not need to be installed - it can be run as a standalone program and will con-nect to any of the HLU Tool relation database formats, including Access, SQL Server, PostgreSQLand Oracle (although connections to PostgreSQL and Oracle have not been tested). However, itmust be run by a user with sufficient database permissions to drop, create & alter tables and todelete, insert, update and select data in the target relational database.

To run the program:

• Download and unpack the latest .zip release file (see Latest Release (page 81)).

• Download the latest scripts from GitHub (see Latest Scripts (page 81)).

• Ensure that any scripts to be processed are placed in the ‘Scripts’ folder directly under theprogram path.

• Double-click the program to start it.

• Click Connect to connect to the target database to be updated.

• Follow the instructions to select the connection type and target database.

• Click Ok once the connection has been established.

• Click Proceed to process the script files found in the ‘Scripts’ folder.

• Progress bars and a text window will show the progress and success of the processing.

• Click Close once all scripts have been processed.

Tip: If you have more than one target HLU tool database, copy the script files from the‘Scripts/Archive’ folder back into the’Scripts’ folder and run the program again - connecting tothe next database to be updated.

82 Chapter 6. Database Updater

CHAPTER7

Optimisation

The design of the HLU Tool has been optimised as far as possible and there are no known simpletechnological enhancements that can be made to significantly improve performance.

It is important therefore to ensure performance is optimised elsewhere wherever possiblethrough user configuration and operation. The following suggestions show some simple ap-proaches to improving performance.

7.1 GIS Optimisation

7.1.1 GIS Layer Indexing

Attribute indexes Certain functions in the HLU Tool will perform attribute searches on the GISlayer, typically using the INCID and/or TOID fields. It is, therefore, recommended thatindividual attribute indexes are created on both of these fields.

Spatial Indexes ArcGIS users that frequently zoom in and out using the GIS application whilstthe GIS layer is visible, when reviewing or editing using the HLU Tool, then it may beadvisable to create one or more spatial indexes on the GIS layer to improve performancewhen ArcGIS draws the layer. The choice of scales will depend upon the number, size anddensity of the features in the GIS layer and the preferred working practices of the users(e.g. if they zoom in and out frequently rather than panning).

7.1.2 Creation of ArcGIS Map Document or MapInfo Workspace

An ArcGIS Map Document (.mxd) or MapInfo Workspace (.wor) must be created for use withthe HLU Tool. This should be optimised to ensure maximum performance, which should includethe following:

• Ensure that the .mxd or .wor file only contains one copy of the HLU layer (orone copy of each HLU Layer if split into separate layers).

83


• Ensure that the .mxd or .wor is saved at a sensible zoom level such as 1:10,000scale rather the full extent of the HLU dataset (to save time when opening).

• Add zoom layering to datasets so that detailed datasets such as the HLU layerand aerial photography are not displayed at smaller scales than necessary. Rec-ommended maximum extents are shown in the table below):

Table 1: Recommended maximum GIS extentsGIS Application HLU layer Aerial PhotographyArcGIS (Do not display out beyond) 1:24,000 1:10,000MapInfo (Max Zoom) 7.5 km 3 km

7.1.3 Tips for ArcGIS and MapInfo users

It takes a significant length of time for ArcGIS or MapInfo to draw or query an entire HLU GISlayer, therefore care should be taken when using certain functions. The following tips may helpavoid some performance issues:

• ‘Zoom to Selection’ is useful for identifying the habitat polygons on the map,however if zoom layering is not used it may take a significant length of time todisplay the result depending upon the number of polygons selected and theirgeographical distribution.

• ‘Select by Attributes’ performs complex queries and selects the results in theGIS window. If a large number of results are returned, it could take a long timeto select the spatial polygons in the GIS.

• If the GIS layer is taking a long time to draw, pause or cancel the drawingusing:

– ArcGIS - Click the Pause button in the bottom left corner of the map win-dow

– MapInfo – Press the keyboard Esc key

• For ArcGIS users, if an ArcMap error occurs when the tool is used, check thatthe HLU ArcMap Extension is enabled in ArcGIS via Tools –> Extensions. SeeEnable HLU Extension (page 11) for more details.

• For ArcGIS users, we strongly recommend that the HLU layer is stored as a filegeodatabase or personal geodatabase.

Caution: The tool will be significantly slower if the HLU layer is stored as ashapefile due to the limitations of the file format.

• For MapInfo users, ensure the Cosmetic layer is editable when digitising a poly-gon for a split operation. The polygon will be added to the layer much fasterthan if the HLU layer was used and also removes the possibility that the poly-gon is not deleted after the split.

84 Chapter 7. Optimisation


7.2 Database Optimisation

7.2.1 Database Type

The HLU Tool is designed to work with both Microsoft Access and SQL Server databases (otherdatabases are supported but have not been tested).

Microsoft Access is a versatile desktop database application, but does not have the same per-formance as other database systems. Database management systems such as SQL Server areoptimised to handle large data volumes and complex queries, and can be significantly faster.

It is therefore recommend that the HLU Tool is used with SQL Server or a similar databasemanagement system as this will improve performance when filtering the database records andupdating attribute data.

7.2.2 Local vs. Network Storage

It is important to remember that application performance will depend upon the data transferspeed. Data stored locally on a single computer will provide good performance, but will limitaccess to the data to a single user. Data stored on a network drive is accessible to all users, butperformance will be limited by the speed that the data can be transferred across the network.

7.2.3 Data Management

Only one copy of the database and the GIS layer should be used to avoid data becoming cor-rupted. If multiple copies are used e.g. an ArcGIS layer and a MapInfo layer or an Accessdatabase and a SQL Server database, changes to one GIS layer or database will not be presenton the other, causing a mismatch between the attribute and spatial data.

Habitat data must not be edited directly in either the database or the GIS layer. Any modifica-tions made outside the HLU Tool could cause data corruption particularly if unique identifiersare altered.

However, if additional entries are required in the lookup tables, these may be added to thedatabase directly. It is essential that the structure of these tables is not altered and we rec-ommend that any updates to the data in these tables are carried out solely by the databaseadministrator.

7.2. Database Optimisation 85


86 Chapter 7. Optimisation

CHAPTER8

Appendix

8.1 GNU Free Documentation License

The full GNU Free Documentation License can be viewed at ‘www.gnu.org/licenses/fdl-1.3.en.html <https://www.gnu.org/licenses/fdl-1.3.en.html>‘

87


88 Chapter 8. Appendix

Index

AAcknowledgements, 6

CConfiguration

Database Connection, 13Exports, 31GIS Connection, 19Habitat Class, 26Habitat Type, 27IHS Habitats, 28Legacy Habitats, 29OSMM to IHS cross-reference, 30Processes, 25Sources, 24Users, 22

DData Tables, 51

History, 60Incid, 57Incid_BAP, 58Incid_IHS_Complex, 58Incid_IHS_Formation, 57Incid_IHS_Management, 58Incid_IHS_Matrix, 57Incid_MM_Polygons, 61Incid_OSMM_Update, 60Incid_Sources, 59Relationships, 70

Database Updater, 81Instructions, 81Requirements, 81

EExport Tables

exports, 68exports_fields, 68

Exports

Export Fields, 31Export Formats, 31Field Formats, 32

IInstallation, 11

LLicensing, 6Linking Access to SQL, 36

Creating a DSN, 41Updating Linked Tables, 47

Lookup Tables, 63Local Flags, 65lut_legacy_habitat, 64lut_osmm_ihs_xref, 65lut_process, 64lut_reason, 64lut_sources, 64lut_Users, 63Sort Order, 65

OOptimisation

Database, 85GIS, 83

SSystem Requirements, 9

UUseful Links, 6

89

HLU Tool - Technical Guide

Documents

Transcript of HLU Tool - Technical Guide