Once again, the annual Hexagon conference “HxGN LIVE” took place in Las Vegas and the IMGS team attended to see the latest developments from the Hexagon family.
The conference is centred around the theme of Shaping Smart Change in 2018 and beyond, and this year (as ever) the conference didn’t disappoint with a range of exciting developments that will change the geospatial industry.
A highlight of HxGN live for me, is attending the keynote presentations from each of the divisions. At the keynotes, each division’s president outlines their vision for their industry and unveils some new toys as well!
I was lucky enough to spend some time with the Safe Software team in Vancouver last month. I was there for our annual partner summit plus the Kick off of the FME World Tour 2018. The partner summit was opened by Don & Dale, and they spoke about the exciting new innovations Safe Software are currently working on, and their plans to grow the Safe team over the coming year. I can’t give much away, but there are some very exciting functionality updates coming down the line. I also got a chance to view the new building, Safe Software are moving to later this year.
Don & Dale have secured the top three floors plus a roof top garden space for the Safeer’s with 360-degree views of Surrey. It was an amazing four days in total and the FME World Tour event was the highlight of the trip.
The new version of FME Desktop & Server was heavily discussed over the course of the few days, and I’m happy to say the team at Safe Software have done it again! They have listened to all the global feedback, and FME Desktop & FME Server 2018 are packed with new and useful features to help you wrangle your data.
With the list of supported formats up to 400 with this latest release, I wanted to do a quick top 10 post on some of the new functionality ahead of our FME World Tour 2018 event in Dublin this May 17th.
I’ll leave links below to the full list of updates in FME Desktop & FME Server 2018 plus how to register for the FME World Tour event. If you’d like to upgrade to FME 2018, you can do so HERE.
Top New Format Reader/Writers
Microsoft PowerPoint Writer
Microsoft Word Writer
PDF Reader (2D/Raster/Text)
Esri Geodatabase Mosaic Dataset Writer
Esri Indexed 3D Scene Layer Writer (i3S)
Apple Venue Format Reader & Writer
Top New Transformers
HDFSConnector (for moving files into and out of Hadoop)
It has often been said, PDF is where data goes to die. Despite this, it is still a go-to format for most organisations. In FME 2018, a new PDF Reader means you can extract rasters, vector data, and text from PDF documents. In the below example, the image on the left is PDF data viewed in the FME Data Inspector. On the right PDF data viewed in Adobe Reader.
I’m very excited about the future of indoor mapping. Now organisations have the ability to create indoor maps for venues like shopping centres, campuses, airports and events. No longer will we be losing our friends at concerts or large events.
With FME 2018 you can move data into Apple Venue Format from applications like AutoCAD, Revit and Tririga. Below is an indoor map of YVR airport, viewed in the FME Data Inspector. If you are interested in indoor mapping and would like to learn more, Safe Software recently did a pod cast on indoor mapping you can listen to HERE.
Generate Microsoft Excel, Word and PowerPoint
I know this will be one of the most popular functions in FME 2018 for most users. The ability to push out data in a friendly styled format like Word or Excel, and enrich it geospatial data will be a huge wow factor for many people.
The new transformers let you style and output your data for Excel spreadsheets, Word pages, and PowerPoint slides. The example below is a map of Canada written to PowerPoint with FME. Each province is its own polygon with a label. The image to the left is a styled Excel sheet, I have also seen the below produced with geotagged images.
Create 3D Web Visualizations
It should come as no surprise that 3D is making a comeback. In fact, I was in a meeting recently and we were discussing 5D, due to Hexagon Geospatial’s recent acquisition of Luciad. Luciad’s visualisation technologies support live connections to dynamic sensor feeds in a 3D environment. The result is a 5D digital reality which allows the user to visualise and analyse real-time geospatial information. It is as amazing as it sounds, like it was from a Marvel movie! Regardless of the amount of D’s you have, these technologies benefit all kinds of applications from public safety to smart cities to defence and intelligence.
Empowering individuals or groups, with the ability to spatially understand an area of interest in all three dimensions, is very powerful. It allows for superior visual analysis and informed decision making. With this latest update, you can create 3D visualisations that anyone can view in a web browser, with new support for Esri I3S and glTF for Cesium. In the below example, you’ll see on the left every building in NYC loaded as Cesium 3D Tiles, and on the right with textures added, written to I3S and loaded in ArcGIS Earth via FME.
Partial Workspace Runs (Tech Preview)
I see this as a very useful new feature, you’ll save so much time when authoring workflows with this new functionality that enables users to run selected parts using cached data. You can now get immediate feedback when authoring FME Workspaces by running only the portion of the workspace you’re focused on.
FME Python API
With the success of the FME & Python training course last year, I wanted to call out this update – a documentation overhaul now makes the FME Python API much easier to navigate and learn from.
Workspace Viewing in FME Server (Tech Preview)
You can now view your workspaces directly in the FME Server interface, I see this functionality growing over the coming years to make FME Server a standalone product for creating, editing and publishing workspaces.
I know this is only a cosmetic change, but I really like the look of FME Server in Dark Mode. In Safe Software the office has been divided into the “Light & Dark Side of FME”.
I have only scratched the surface of all the new features and functions of FME 2018. If you would like to see more information on the update you can click on the links below or better yet join us at the FME World Tour in May.
In today’s big data world, we have seen an explosion of data from structured data sources such as databases to unstructured sources such as twitter feeds. This data growth has led to a movement away from the old ways of relational databases to NoSQL flat databases such as MongoDB.
This data growth has also driven the need for distributed systems such as Hadoop to drive the processing of these massive datasets, and where on-premise servers can’t handle this processing organisations are moving to the cloud for greater elasticity and scalability.
But with this data growth, we must ask the question are we any better off? Is this data helping or hindering decision making?
I feel this data growth is off course of great benefit but for us to benefit from this data growth we need to understand our data properly. To understand our data, we need to have metadata or data about our data.
Too many analytical projects fail because of a lack of understanding of the true single source of data, whether it be a single view of a customer or a single view of an asset. With data now stored in many different systems, formats and locations (on premise and in the cloud), using Metadata to better understand this information is essential.
Coming from a geospatial background I have talked about metadata for some years now and at one stage the EU INSPIRE initiative was the driving for force for organisations to manage their Geospatial data better and develop proper metadata catalogues.
Unfortunately INSPIRE never really achieved what it planned to in my opinion but from the 25th of May there will be new GDPR legislation which will drive us all to manage our data better.
It is easy to scoff at the EU, and to parody the great Monty Python Life of Brian maybe ask ourselves what have the Romans EU ever done for us – except for the money, roads etc.
And it maybe its unpopular to say but most EU legislation is of benefit to the citizens of Europe and in my opinion GDPR is no different.
As we have seen in the last month with the Cambridge Analytica data breach scandal, protecting customer data is of utmost priority through proper data governance (and if you don’t understand your data – how can you protect it). Also, unlike INSPIRE, the GDPR regulations have teeth and any company who is in breach of these regulations will suffer large fines.
But managing metadata isn’t just about meeting GDPR or any other data regulations (and believe me more are coming). If we can understand our data better, we can make better decisions. By making better decisions our organisations can grow sales and cut costs through improved efficiencies. Before you make any decisions on new Big Data analytical projects, ensure you understand your data by implementing a proper meta management strategy in your organisation, and the ROI for your Big Data projects will increase substantially.
If you would like to hear more about our metadata solutions, please listen to my Irish Tech News podcast https://buff.ly/2G22pUM or attend our up coming webinar on how we can easily handle GDPR subject data access right requests using our metadata solutions https://bit.ly/2Fy5gb6
Using FME daily means you pick up on tips and tricks to make your daily tasks run quickly, smoothly and generally just make your FME life a bit easier. This blog is a follow up to last month’s time savings tips part 1.
Tips for Inspecting data
Inspecting your data is a key part of any workspace you develop, with every change you make its good practice to check your output. So it stands to reason FME has a number of tools available to you, all you need to do is use them.
Run with Full Inspection
Do you ever run your workspace, wait for it to finish only to find it’s taken a wrong turn somewhere? Now you need to find out where things went amiss, run with full inspection is the solution for you. This setting will cache the data at each transformer link. Each cache can then be shown in the Data inspector by right clicking on the count.
This feature is great for debugging, no need to keep adding inspectors to source the issue, but don’t forget to turn off this functionality when you are done as it slows down performance and takes longer to run than a normal translation.
Redirect to Inspection Application
When you are still in a testing phase then use “Redirect to Inspection Application”. This means you don’t alter your destination dataset and your output goes directly to the Inspector. You can run your process as many times as you want until you get it just right, the data is never written until you turn off this option. Especially important when you are making edit within a database and you only have one chance to get it right.
One Click Insect
Your readers and writers now allow you to inspect your data with ease, right click on any Reader/Writer to use the inspection button or open containing folder. No excuses for not examining your output it’s easy!
The terminator transformer stops a translation when it detects non-valid situations or input data conditions that should not exist.
This is to stop the output of data which may impact your results, for example you could filter the data’s geometry and terminate if you find points in a dataset you expected to be linear. When a feature enters this transformer, the translation immediately stops and an error message displays to notify you that the data conditions are not expected.
To redirect the Terminator data to an Inspector instead of stopping the translation, you can set this option in the Workspace Properties in the Navigator. This allows you to view the unexpected data during development of your workspace and then you can switch off and reverts back to terminating the translation when expected conditions are met.
Once you are inspecting the data its quite useful to compare to a background map, don’t forget the Data Inspector now provides this functionality under Tools>Options>background maps.
You have a number of options ranging from open source mapping like open street map to the ability to connect to your own AGOL Map Service or Web Map Service. But remember your data will be reprojected to display in the coordinate system of the background map. So don’t get confused like I do….everytime! The actual coordinate system is always specified within the feature information.
Sometimes you can add a file with many layers like a .dgn or Geodatabase. Nothing gives you a bigger headache than having to reconnect all those layers to your writers, particularly when they are located the far side of the canvas. The Feature Type Connections window, found under View > Windows allows you to connect readers to writers with ease. I don’t use it often but when the need occurs it’s a great time saver.
The most popular transformer the Tester is much more than what it seems. You should have noticed by now it has received an update in the last few releases. If not, then pay attention as this can be a huge time saver when performing data quality checks. There are now Operators to filter data on such as Attribute is Null or Empty. New to 2016 is the operator “Attribute has a value”. This means that without knowing all the possibilities of how the attribute could not have a value we only need to check if it contains a value in order to extract all those missing records.
Finally don’t forget you always have options! It’s important to customize your workspaces to your own needs so take 2 minutes to check out your options under the Tools Menu > FME Options. Here you can create a toolbar to your own specifications.
As well as control how your workbench functions, default paths, database connections and how your translation is run. There are plenty of choices here so they are worth reviewing.
Thanks for reading! Keep an eye here for next month’s blog.
Using FME daily means you pick up on tips and tricks to make your daily tasks run quickly, smoothly and generally just make your FME life a bit easier. This blog has been a long time coming, for the last year I have diligently kept note of some of the tools I use in Desktop that save time and because it’s nice to share I am departing these tips to you.
Functions may seem complex as they allow you to delve into the heart of FME’s core and fly solo so to speak. Using them means you are now stepping out on your own and calling functions that are in use in many of the transformers and that seems a bit daunting.
But this really is a smart move on your behalf, by using Feature Functions you can build processing directly into your attribute creation and save yourself the effort of placing multiple transformers. Really you can do A LOT with the text editor in FME, use it to build your queries and send them directly to the engine yourself instead of relying on transformers. When you use FME Functions as part of your attribute manager or attribute creator each resulting value is output as an attribute.
For example, @Area function calculates the area of a polygonal feature eliminating the need for the AreaCalculator. The @Count provides a mechanism for generating unique numbers for features during a translation eliminating the Counter. @Timestamp returns the current time and date, so you guessed it no need for a Timestamper.
Here are some of the common FME Functions I use in my workspaces, why don’t you try them out next time and see how simple it is to develop your own results without the use of multiple transformers.
Also for those experienced FME users out there did you know you can copy and paste a transformer into an open Text Editor and review the content, this allows you to see what really is at work inside the transformers you use.
The ability to save workspaces as templates was added in 2011 though it still isn’t as widely used as I would like! A template is basically a saved workspace, with the inclusion of the data required to run. Using a template means you can bundle everything together and provide to another user….very handy for sharing workspaces with your colleagues.
These templates are given the extension .fmwt, using the template is very simple just open it in Workbench. If data was included in the template file then it will be extracted, stored to a temporary location and the workspace paths updated automatically making it ready to just hit run.
There are many possibilities with templates, they can be great to give out to other FME users as predefined translations. Or when looking for support from the team at IMGS if you provide us with a template you will certainly rise of the top of our to-do list and receive quicker solutions!
While on the subject of templates, if you are reading data from a database and the end user of the template does not have access to this data source then it’s time to introduce a recorder transformer.
The recorder is a nifty little transformer that saves a copy of all the features that enter it and writes into a FME Feature Store file. It’s really simple to use;
In your workspace identify the data you want to export.
Add a Recorder transformer
Connect the Reader feature types you need to export to the Recorder transformer
Run your workspace and the data is stored to the FFS file at your specified location
Now you can send a copy of your workspace and the FFS file. The end user can then add a Player transformer to your workspace and play back your recording – as if they were reading directly from your database. Everyone wins!
Most people probably know everything they need to know about the data fanout, it has been around a long time. But when it comes to listing features that save you time and effort I have to include the fanout. This is a way you can split your output data into separate layers/files based upon an attribute value.
It’s easy to set, just go to the writer properties right click and pick an attribute value. So your data output will be divided on the fly as it writes to your destination.
For example, I want to create a set of contours from my LIDAR dataset. But I want to have a separate output file for every elevation. If I set the output to fanout by the value of the elevation my results will be quite different.
Instead of one file containing all contours your destination output will now contain multiple files one for each elevation level.
People always say there are no shortcuts in life, well that’s not quite true. Here are some custom keyboard shortcuts which I promise will make your life easier, no risk involved!
Add New Bookmark
Zoom to 100%
Another one of my favourites is the quick change from zoom in to select. Try it yourself, Zoom in and right click this will resets the cursor back to the select function. It’s the simple things in life.
Stay tuned for the follow up part two blog of Time Saving Tips in FME which will be published next month.
For a new user looking at FME’s hundreds of available transformers, many with some interesting and unusual names, locating the transformer you need to get the job done can often feel like a difficult or daunting task.As an FME professional and Trainer I often get told in panic “I don’t think I will know what Transformer to use, there are so many!!” and that is true. In reality you can still be a proficient user of FME without knowing the ins and outs of every single Transformer. That is, if you arm yourself with the most useful ones!
Transformer knowledge comes down to experience. The more workspaces you create and projects you complete the larger your catalogue of Transformers becomes. To get new users started I would like to share with you a list of some of the unsung heroes of the transformer world, these are some of the tools I really couldn’t live without.
I am most often asked about how to quickly move spatial data between different coordinate systems. The answer is very simple, use the Reprojector. FME’s coordinate reprojection capabilities solve the common requirement to reproject data accurately and seamlessly. All you are required to do is place the transformer somewhere within your workflow.
This transformer works on both raster and vector data, It provides the option to reproject data with a library of over 5000 coordinate reference systems. This gives you the ability to reproject data with the click of a button eliminating all those coordinate system headaches you face as a spatial data user.
The Clipper is an excellent transformer for creating datasets that are relevant to your needs. Often users have national datasets containing more data than they actually require. The Clipper takes a number of clip boundaries and the number of features to be clipped and splits the data into features inside the boundary and features outside the boundary.
As the output is split into these two possible groups the Clipper allows you to keep the data relevant to your needs while disregarding the data that falls outside your defined boundaries. It is a great way of cleansing your datasets to only contain the data you will use, meaning any further analysis on these datasets are correct and more meaningful to your area of interest.
If you ever come across the need to provide a unique ID to the data running through your workspace than the Counter is the most simple way to achieve this. This transformer adds a numeric attribute to your features by assigning each feature an incremented value as it counts the features passing through.
You can also specify if this counter sequence is unique throughout the whole workspace by using the same sequence on multiple counter transformers or using a unique sequence to that single counter. This transformer means you can create unique values without using any complex expressions, calculators or calling database sequences.
Another key quality control task it to check your data for duplicates, sometimes no matter how ‘perfect’ you think your data is, an unwanted duplicate can creep in during a processing task. FME gives you the ability to ensure that your data is clean by eliminating all those duplicate objects so only one remains. The answer comes in the form of the appropriately name DuplicateRemover. This transformer allows you to select one or multiple key attribute values that you require to be unique. Any key attribute value that has already been encountered in the DuplicateRemover is sent to a separate port labelled as a duplicate.
Allowing you to maintain the data stream coming from the unique port, giving you confidence in the quality of your output data.
Lastly, users are often tasked with the requirement to provide labels with their output data. There are a number of options in FME for labelling the most basic being the LabelPointReplacer. This transformer enables you to quickly configure labels and rules that will dictate how labels appear.
This transformer replaces the geometry of the coming feature with a label point while also giving you a number of options to define the label value.
If the feature was already a point, it is turned into a text feature at that location.
If the feature was a line, it is turned into a point text feature at the midpoint of the line and rotated parallel to the line.
If the feature was an area feature, the resulting point is placed in the interior of the feature.
The LabelPointReplacer removes the hassle of creating labels for your spatial data and should be your first stop when tackling label requirements.
These are some of the key transformers I regularly use when tackling data problems. Hopefully it will give new users a head start on which of the transformers they may want to try out in the future. But don’t forget if you are ever in doubt or lost on what is the transformer for the job;
Use the associated Help File– Every transformer you select will link to the help file which pops up in the help tab in your workspace. If you are unsure of how to use a certain transformer the help file gives you detailed information on the use of the transformer and how to configure the parameters, input and output ports.
Use the Transformer search functionality in the Transformer Gallery to pinpoint your possible transformer options. This searches in both Transformer name and description so your search term may be the exact name of a transformer, or it may be a general keyword referring to functionality.
If all else fails then just try it out, trial and error can often be the best option in understanding exactly how each transformer effects your data. Don’t worry you won’t do any damage to your dataset and can run your workspace as many times as you wish. Just connect an inspector transformer to the end of your workflow instead of connecting it to the writer and you can keep testing until you get it right!
Also help is always at hand just contact us at firstname.lastname@example.org for FME related queries we are happy to point you in the right direction when you are lost in a maze of Transformer options.
Author: Vijay Sambhe, Senior Applications Engineer at IMGS
Hexagon’s Geospatial Portal – Locate a coordinate in the Map
Hexagon’s Geospatial Portal is a full-featured, configurable and customisable thin client application that can be used for finding, viewing, querying, analysing, and consuming geospatial data published by Hexagon Geospatial products and/or other standard-based web services. The portal can be accessed by multiple users via web browsers as it is installed and configured on a web application server.
When the Locate button is clicked the map zooms in to the location and also it shows the message as shown below.
Plenty of help and code examples are available on Hexagon’s Knowledge base here.
Author: Ciarán Kinane, Applications Engineer at IMGS
Hello everyone and welcome to this month’s blog. This month we are going to focus on map based asset management using hexagon’s GeoMedia Smart Client.
Nowadays people can see the value of GIS within your organization, but they often lack the expertise to run a desktop GIS. Online maps delivered by a web GIS solution are helpful, but these implementations in many cases do not support specific geospatial workflows required by the user. They also may not provide some of the main functionality of a desktop GIS such as snapping etc. Today, users look more and more for market-focused workflows specified to their needs.
The vast majority of people who could really benefit from geospatial data need more than what a lightweight, browser-based web map can deliver. Likewise, they don’t have the time or expertise to learn a technically demanding desktop environment. These users are eager to geospatially accelerate their business workflows and ignite the power of GIS. This requires a new approach, working outside limiting departmental deployments of GIS that have plagued organizations for decades.
This is where GeoMedia Smart Client comes in. It provides a platform for users which is easy to use and does not require the same training or knowledge required by a desktop GIS. Smart Client is delivered to users via a browser and runs on the client machine using Java. The thing I really like about Smart Client is that it only requires 1 license whether you have 5 users or 500 users running concurrently. Smart Client has the ability to open up GIS to your whole organisation.
Here in IMGS we have provided separately configured asset management systems to both Shannon Properties PLC and Sligo County Council using Hexagon Geospatial’s Smart Client. Out of the box functionality provides, measurement, dimensioning, snapping, redlining and configurable printing, but the real power in Smart Client is the ability to create interactive workflows to meet the user’s needs.
For Sligo County Council we used Smart Client to replace their fully customized map based asset management solution. Using Workflow Manager, which comes with the professional version of Smart Client, we were able to create an Asset Search workflow.
The Asset Search has the ability to filter on multiple attributes within the asset database. For each entry returned in the search, we have the ability to map to, edit or view that asset and sort by any attribute. The asset list can also be printed to csv, excel, word or PDF and can include XY coordinates of the asset, criteria which are a requirement of the Office of Public Works in Ireland.
We also created a workflow which allows the user add a new asset to their system. When the user selects the ‘NEW’ button they are presented with a form whereby the user can enter details pertaining to the asset. The workflow can have many rules to prompt and aid the user with the compiling of asset details. For example, the user can enter no more than 5 digits as the asset number, or the asset ID can be created based on the asset number with a prefix coincident with the asset type specified in a drop down box.
Once the user has entered in the minimum details, they can begin to digitizing the asset in the map window.
While digitizing, the user is afforded many features which would not be available within a web based mapping solution, such as snapping to vector data (as seen on the image below), dimensioning and the ability to scroll the map while in capture mode etc.
Various geometry rules can be applied at this point to guide the users. For instance in the Shannon Properties project they requested some rules which would not allow the user to capture the boundary of a building outside the boundary of a particular site. Another rule requested was to not allow the user to capture the boundary of a site outside a particular Landbank feature. These rules can be applied and guides set in place to direct the user reducing anomalies in the data and standardising your digitizing processes.
Once captured in the map window, the user is brought back to the workflow where they can enter additional details into attribute fields. The form is comprised of tabs along the top which contain different aspects of the asset such as terms of leasing etc. The area of the polygon is calculated in the database and added into the form. A specific request from Sligo County Council was to show the area measurement in hectares if the area was above 5000 meters squared to have it shown in squared meters otherwise.
The workflow can be configured to calculate and report such things as the townland or townlands the asset was captured in via a spatial query. Adjoining streets or roads can be also calculated in this way and a report can be produced and printed with the associated planning applications.
Other features within the workflow are links to external applications within or outside the organisation such as iDocs., the ability to upload files or images, and the ability to add notes and flag one of the notes so that it then appears as the tooltip for the asset in the map.
Various actions can be added to the map which can directly open any workflow, for instance the image shows the ability to be able to copy an existing land parcel to one of your assets just by highlighting the land parcel and right clicking the mouse.
Smart Client has some neat features around bookmarking, you can bookmark an area of interest, save it and then e-mail it to other users or you can share the bookmark across your organisation so it is available to all users.
Access to Smart Client can be based on a windows user login and it not only has the ability to restrict users to the application but it also has the ability to restrict windows users to Smart Client Projects, Features, Workflows and even functions within a workflow. So where one user can have the ability to edit an asset other users can be restricted from doing so within the same project or view.
In addition Smart Client can be configured to launch standalone workflows, for instance if a user wants to just add attribute data to an asset they can do so by launching a standalone workflow without having to launch the GIS application.
Overall, Smart Client’s most prominent capabilities are the ability to supply a GIS application to everyone in your organisation without having to license each seat. With a shallow learning curve and the ability to provide customized workflows there is something for everyone in the audience!
For further information on Smart Client please feel free to contact me email@example.com
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.