imgsblog

Regular Expression Dialogue

Regular Expressions or understanding search Hieroglyphs!

Regular Expressions or Understanding Search Hieroglyphs

Regular expressions are a sequence of characters that specifies a search pattern. Usually, such patterns are used by string-searching algorithms for “find” or “find and replace” operations on strings, or for input validation. They are therefore incredibly powerful when it comes to validating data, ensuring data matches an expected structure or as we will see below in aiding tasks such as address cleaning and geocoding.

Whilst the example below will focus on their use in FME, the concepts discussed will apply to any tool or programming language that supports them, so even if you are not an FME user hopefully you will find the following discussion helpful.

Recently we were met with a support call that required removing the numeric digits at the beginning of an address. The problem was that the number of digits varied. For example, the data had the following, 9 OAKDALE, 23 AL UISCE and to complicate it further, some entries were similar to 44A CHERRYORCHARD HEIGHTS where the 44A would have to be removed. The end result was to return only the text that fell after the space (OAKDALE, AL UISCE, CHERRYORCHARD HEIGHTS).

Our initial thoughts were that a regular expression would be involved in solving the problem. These are a somewhat hidden, and intimidating bit of functionality within FME that can be hugely powerful once you have gained a little mastery of them. Our initial attempt to use them didn’t give us the output we were hoping for. The digits were stripped from the beginning of the value, but a space remained and worse 44A CHERRYORCHARD HEIGHTS became A CHERRYORCHARD HEIGHTS.

We went back to our workspace (Fig 1) and set about tweaking it to suit the issue at hand. The data was being fed into a StringSearcher transformer that had a Regular Expression parameter (Fig 2). We had found the initial expression from an online search and knew what this regular expression was doing from the output but we could not decipher what each symbol was doing or how to alter the expression to suit our needs. Our initial attempt was promising and looked like it was close to what we were after, so we decided it was time to get our head around the regular expression!

Regular Expressions Figure 1
Fig 1: FME Workspace
Regular Expressions Figure 2
Fig 2: StringSearcher Transformer Parameters

The first port of call was to look at the help for the StringSearcher parameter and try out the regular expression editor in the StringSearch transformer. Like the text editor which can be accessed in transformers that allow for text input this editor dialogue appears where you can insert a regular expression.

Regular Expression Dialogue
Fig 3: Regular Expression Dialogue

Our initial attempt let us see that we were matching the string and started to give us an idea of what the individual elements of the expression were doing but it was still quite confusing to see what the expression as a whole was doing. We then tried Michael Lovitt’s excellent online Rublar expression editor at https://rubular.com which let us see that our expression was actually matching two groups from the single match shown in FME’s editor.

Regular Expressions Rubular.com
Fig 4: Rubular.com Matched Groups

Before we show you how we solved it we want to give a quick primer with regards to regular expression and hopefully take someone that has never used them before and make them (slightly) more comfortable with attempting to use them in the future. We’ll dive straight into the deep end with the expression from Fig 2 and the break it down.

Regular expressions are used for pattern matching. It is a sequence of characters (or symbols) that forms a search pattern and is generally applied to string matching. Let’s take a look at that expression in Fig 2:

^([0-9]+)(.+)$

This might as well have been hieroglyphics but when you break down the expression it actually quite simple. This expression states that the string must begin with at least one or more digits with each digit being from 0-9. So this can be 1, 26, 45, 111. The ^([0-9]+) is the part of the expression that tells us this. ^ means at the start or begin with. This is followed by a set of parentheses ( ). The parentheses indicate a capturing group. This is group 1. The next set of parentheses would be group 2 and so on. Don’t worry if capturing groups do not make sense at this stage, when we return to FME to talk through and run an expression it will be a lot clearer. Inside the first group we have [0-9]+, the [0-9] is a character set of 0, 1, 2, 3, 4, 5, 6, 7, 8 ,9 and the + means one or more. So if we take ^([0-9]+) you can see that this means, match strings that begin with one or more numerical digits. If the + was not present, this would mean that the string could only begin with one digit from the character set.

The next capturing group (.+) had us scratching our heads at first because it just contains symbols. The dot or period simply means match any character, like a wildcard. We have already met the + symbol. This group says match at least one character to any amount of characters. Again, if we removed the + it would only match one (any) character. The $ means the end of the string. So, match one or more characters until the end of the string.

Let’s look at that expression again in its entirety. The string must begin with at least one digit or more followed by at least one or more characters until the end of the string. We will use 9 OAKDALE as an example. This begins with a single digit so the initial criteria ^([0-9]+) is met. 9 belongs to group 1. This is followed by a space, which is also a character, and followed by seven more characters until the end of the string. This satisfies the (.+)$ and places OAKDALE(with the space in front of it) into group 2.

In the FME workflow in Fig 1, both groups will be output as the matched result recreating what we had initially! As we are using groups, we need to create a subexpression List under the transformers Advanced settings. This will output a List with two elements, one for each of our subgroups, this includes both the found string but also its starting position within the original string value.

Fig 5: Matched sub groups as a List element
Matched sub groups as a List element b

So now we understand how FME is processing the current regular expression and can identify the sub groups how do we alter the expression to produce the desired output? We will need to look at a few more symbols and what they can do. To do this we are going to switch the focus away from the problem we are trying to solve and use another example relating to Irish mobile numbers. Take a deep breath and have look at the expression below:

^(\(08[3,5,6,7]\)|08[3,5,6,7][ -]?)[0-9]{7}$

What is this madness? This expression is verifying how an Irish mobile number can be entered let’s say into a form. The number must begin with 08 followed by 3, 5, 6, or 7 wrapped in parentheses, OR begin with 08 followed by 3, 5, 6, or 7 followed by an option space or hyphen, and then followed by seven digits.

Before we go through the new symbols we are meeting here we want you to open https://rubular.com/ and copy and paste the expression into the top textbox. In the lower textbox enter (087)1234567, the entered text should highlight in yellow meaning that it meets the criteria set out in the expression (Fig 3). Next try 0871234567, then 087 1234567, and then 087-1234567 and note that these are all accepted by the expression.

Fig 6: Rubular regular expression validator

Now try (087) 1234567, and then (087)-1234567. These will not highlight because they are invalid. Here is a list of other violations for you to test:

087-123-4567     hyphen separating after 123

087 123 4567       space separating after 123

084-1234567       4 is not a valid third digit

(081)1234567     1 is not a valid third digit

082 1234567        2 is not a valid third digit

0881234567         8 is not a valid third digit

Right, so we can see how a regular expression can detect a valid and invalid entry. Let’s go through this example and then use what we learn to fix the expression in our workspace. We know that ^ means at the start, this is followed by a set of parentheses. There is a large chunk within these parentheses. We will first look at everything to the left of the |

\(08[3,5,6,7]\)

The \ is an escape character meaning that whatever symbol immediately follows it should be considered a character literal to be searched for and not a part of the expression syntax. There are two in the snippet above that allows the expression to search for an opening and closing parenthesis. The opening parenthesis must be followed by a 0 and then an 8, so the beginning must match (08. The [3,5,6,7] is a character set. The next digit can only be one of these digits, so we are now at (083 or (085 or (086 or (087. These are then followed by the closing parenthesis. Therefore, our entry must begin with (08[3,5,6, or 7]).

But the | means OR, like a branch in computer programming. We will now look at the expression to the right of the | still within the original parentheses that are not escaped (capturing group). Our entry can begin with what was to the left of the | or the following:

08[3,5,6,7][ -]?

The above reads that our number must begin with 08 followed by 3, 5, 6, or 7, followed by an optional blank space or hyphen. The ? means zero or one occurrence, so after the initial three digits there can be one space OR one hyphen OR no space and no hyphen.

You should now be able to decipher what the first part of this expression is looking for:

^(\(08[3,5,6,7]\)|08[3,5,6,7][ -]?)

Entries that begin with the following:

(083)    OR      083 (followed by a space)       OR      083-     OR      083 (no space, no hyphen)

(085)    OR      085 (followed by a space)       OR      085-     OR      085 (no space, no hyphen)

(086)    OR      086 (followed by a space)       OR      086-     OR      086 (no space, no hyphen)

(087)    OR      087 (followed by a space)       OR      087-     OR      087 (no space, no hyphen)

The next part of the expression:

[0-9]{7}$

This reads seven occurrences of digits from the set of 0-9 and that is the end of the entry. An eight digit or entering a character that was not 0-9 would mean the entry is invalid. Therefore the curly braces { } is a quantifier of how many occurrences of the expression immediately before it should exist. If there was a {3} then only three digits could follow.

Let’s go through the full expression once more and make sure that we have a handle on this:

^(\(08[3,5,6,7]\)|08[3,5,6,7][ -]?)[0-9]{7}$

The entry must either begin with three digits wrapped in parentheses with the first two being 08 followed by either 3, 5, 6, or 7, OR begin with 08 followed by 3, 5, 6, or 7 with an optional space OR hyphen, followed by seven digits, each from 0 to 9. That reads so simple now that we know what we are doing.

We are now armed with what we need to rectify the expression to solve our issue. The problem we are faced with is that we have entries with one or more number of digits followed by a space, or one or more number of digits followed by a letter (44A) and then a space. We want to cut away those and only be left with the text that follows the space. Let’s begin to write our expression. We know that ^ means the start of or beginning of the string so we will start with that:

^

We have an OR situation where the value can have digits only or digits and a letter. We will begin to construct the first situation that will sit to the left of the |:

[0-9]+[ ]

Now to the right of the |:

[0-9]+[A-Z][ ]

We will wrap the left and right of the | in parentheses and put it together to give us the first part of our expression:

^([0-9]+[ ]|[0-9]+[A-Z][])

At this stage we have an expression that is matching values that begin with either one or more digits followed by a space OR one or more digits followed by a one character from the set A-Z followed by a space. This looks good so far and we can simply take the last part from the expression in Fig 2

(.+)$

and append it to our expression.

^([0-9]+[ ]|[0-9]+[A-Z][ ])(.+)$

Our full expression now matches values that begin with either one or more digits followed by a space OR one or more digits followed by a one character from the set A-Z followed by a space, followed by one or more amount of any characters until the end of the value.

We can now enter this into the Regular Expression parameter of the StringSearcher to see if it works. SUCCESS! The output now creates an initial subgroup containing the property number if present and the second group contains the estate name. We can add these as new attributes to our data as shown below in figure 7.

Fig 7: Output within the Visual Preview window

Before we finish up we want to talk you through the entire workspace presented in Fig 8 below. The Reader reads a CSV file consisting of three attributes. The Estate_Name is the one we are interested in and wherein our problem lies.

Final Workspace Fig. 8
Fig 8: Final Workspace

The Reader connects to a StringSearcher transformer where we set a few parameters (Fig 9) as shown below:

Fig 9: StringSearcher Settings

As mentioned earlier the Matched Result attribute will output the same value as Estate_Name because our regular expression matches all of the elements of the string, so it’s considered a match. As we are matching subgroups, we can leave this blank to stop FME creating an unneeded attribute and under the Advanced section we can enter a name for the output list that will contain the matched groups from our expression. Our first group from ([0-9]+[ ]|[0-9]+[A-Z][ ]) is going to be placed in _lstSubExpression{0}, therefore, the 1 , 22 , 101 , 44A , each followed by the space, will be placed in here. Our second group (.+) is what we are most interested in, this contains everything after the space and gets placed in _lstSubExpression{1}.

The results from both ports of the StringSearcher are passed into an AttributeManager which handles both the matched and unmatched results and creates two new attributes representing the House Number and the Street Name. This is done through the use of Conditional Tests as shown in the figure below.

Fig 10: Conditional Tests to create clean output values

The conditional tests, check to see if the first list element has a value and if it contains a property number adds this as the House Number, if it doesn’t which would be the case of the unmatched records, we explicitly declare the output as Null, so we know we do not have data for this field. A similar set of tests are carried out to populate the Street_Name attribute. Our list is well ordered as a result of the matching process, we know this will be the second item in the list we can use the list element order to determine which part of the list to use.

The AttributeManager demonstrates how it is possible to combine a number of steps by allowing us to wrap up the step of creating the attributes, merging those cases without a property number and cleaning up the data flow by removing the lists in a single step.

Hopefully this will have shown how powerful regular expressions can be and might be used to solve some of your future problems.

Just to test if you have learned anything from this post check out the expression below that does the same as our final expression but more compact and without an OR | branch. Does this make sense to you?

^([0-9]+[A-Z]?[ ])(.+)$

Hopefully our journey through regular expressions has shown how powerful they can be as a tool for pattern matching strings and can be used to validate data such as telephone numbers, Eircode’s, or to aid with address cleansing problems as we seen in this example.  

Alongside the examples we’ve touched on you can find online cheatsheets such as can http://www.cheatography.com/davechild/cheat-sheets/regular-expressions/that can help turn the hieroglyphics into understanding.

We will leave you with one final note; although most implementations of regular expressions are extremely similar there can be slight differences, this means small differences in your data can mean the difference between a match or not so it’s always good to thoroughly test your expressions and transformers like the Sampler can create a subset of your data for this.

Finally, here is one last challenge can you see how this slightly modified version of the expression above differs and do you think it’s a better solution?

^([0-9]+[A-Z]*[ ]*)(.+)$

Remote Working During a Pandemic

Remote working in a pandemic

Remote Working during a Pandemic

By Tanya McDonald, Marketing Executive, IMGS

Last year on March 12th the Taoiseach of the time, Leo Varadkar, gave a landmark speech from Washington announcing a raft of Covid restrictions for Ireland, which included the closure of schools and a switch to remote working for those that could.  Little did we know that this would be the ongoing reality for a large cohort of the population over a year later.

Following that announcement, my kitchen table doubled up as a classroom and home office in between mealtimes during the day.  Multi-tasking took on a whole new meaning as I took on the role of teacher alongside my day job!

Remote Working During a Pandemic

Home Schooling Support

In what was a very difficult situation, the schools offered as much support as they could by providing a combination of recorded lessons on Seesaw and Google Classroom, scheduled zoom calls and a timetable of work for the week.  The structure enabled my children to work mostly independently and freed me up to get my work done.  The RTE Home School Hub also provided some respite for an hr a day when I needed time to concentrate.

Transition to Remote Working

I have found the transition to remote working easy enough this past year, as in-person meetings were replaced with Microsoft Teams meetings and day to day operations within IMGS continued as normal.  Once it became clear that remote working was going to be a long-term arrangement, I moved from the kitchen table to an office set-up in my Son’s bedroom with an ergonomic friendly workspace, that my back has thanked me for!

Events move on-line

The pandemic has certainly accelerated the need for change in the way that we work and plan and run events.  As a Marketing Executive in IMGS, a large part of my role involves the planning and execution of events.  As the pandemic spread and the global situation deteriorated further, it became clear that our events would have to move online for the remainder of 2020 and a virtual alternative found.  We went with Accelevents as our virtual event platform and it has proven highly effective in delivering quality virtual events and exhibitions. It has been so successful that we may continue to use it post pandemic, and offer a hybrid mix of events in the future. 

Is remote working the way forward?

When I reflect on the past year, my overall experience of working from home has been largely positive and even more so since the schools reopened again 😊.  It offers a good work/ life balance and I am sure will be viewed as a more acceptable working arrangement going forward.  Personally, I would favour a hybrid approach encompassing a mix of in-office and remote working, this would fulfil the need for social interaction at work and the opportunity to enjoy a bit of office banter and a Friday lunch when hospitality reopens! 

While I write this blog things are looking brighter and hopefully the majority of us will be vaccinated by the Summer facilitating a return to a more normal way of life.

Kieran Massey, I.T. Grad, IMGS

Kieran Massey, I.T. Grad, IMGS

Kieran Massey in the Staff Profile hot seat this week.  Thanks for a fantastic interview Kieran!

Can you describe your role at IMGS?

My current role at IMGS is IT graduate, I started with IMGS back in April 2020, I was lucky to get in with IMGS when I did because of Covid just kicking off, things were uncertain for a spell . My role consists of working with the development team and meeting customers’ data driven needs in developing applications, BIs, generally using Hexagon’s M.App Enterprise Studio so far in my daily work. I would also work daily with Databases, some CSS where applicable and some dodgy Javascript from myself when also needed.

 

What do you like to do in your spare time?

In my spare time I like to play Playstation, big gamer since my dad bought me a Nintendo 64 back in the day. I also read comics from time to time, I have a nephew too so I try to be involved with Jaxson as much as I am able throughout Covid. I do like blowing off steam at the gym, but with Covid its more of a home grown gym, caveman style!

 

What’s the last book that you read?

Apart from comics, I’m reading the Lee Childs ‘Jack Reacher’ collection, I was surprised to find out there is 19 books in the series, I’ve 3 down, a lot to go.

 

What will be the first destination that you will visit overseas when travel reopens and why?

Wow first destination, tough one! My Fiancée and I (Snuck that one in!) have been putting off Greece for awhile and I think after this everlasting lockdown, we owe it to ourselves to go over and visit Greece. Other than that just to be able to go out for some food again with the gang, drinks, all of that social aspect to life, will be nice to have again.

 

If you could do another job for just one day, what would it be?

I think I’d like to try acting! Very random but I am a movie buff so to act for a few years would be cool, play on the Irish accent thing for a while. Make sure I got to be in a movie with Salma Hayek!

 

What do you enjoy most about working at IMGS?

I love the closeness of the team actually, even though we’re remotely working, I can feel how close and connected the team is to one another, and that made coming into IMGS so easily to adapt to also. My last job wasn’t so team driven which was just chaos to be honest, so to see how close IMGS is and to be apart of this family now is special.

I love how every day is different in my job, be it creating data driven customized applications, or BIs for customer needs, every day so far has brought something different and I wouldn’t change that for the world!

Keith Duffy, I.T. Grad, IMGS

Keith Duffy, I.T. Grad, IMGS

Keith Duffy, I.T. Grad, IMGS

Can you describe your role at IMGS?

I work as a developer for IMGS. Since arriving a little over 6 months ago, I have done a mix of both front and back-end development. For my first project after joining, I was tasked with designing and building a new WordPress website for the company, that went live a couple of months later. Since its completion, I have moved on to working on mainly database and form design using a mix of SQL and JavaScript. 

 

What do you like to do in your spare time?

In my spare time, I play competitive First-Person Shooters, mainly Rainbow Six Siege. I built a gaming PC right at the start of the lockdown and it has been a godsend throughout. As there can be no socialising at the moment, I feel very lucky and privileged to have a large group of friends that so I can hop on with in the evenings and play together. When I’m not stuck in front of a computer, I spend my time indoor rock climbing, practicing drums, doing yoga, or jamming music with friends.

 

What’s the last app that you downloaded?

The last app that I downloaded was Waking Up. This is a meditation app by Sam Harris who is a philosopher, New York Times bestselling author and neuroscientist. Over the last number of years, I have gotten quite heavily into meditation. I used a couple of apps before this one, but I love the more mindfulness based, Vipassana style meditations that Sam specialises in. It has really helped me throughout this difficult time, and I would be lost without it.

 

What will be the first destination that you will visit overseas when travel reopens and why?

The first place I will probably get to travel to after everything reopens will be my honeymoon. Although the exact destination has not been decided yet, we have been seriously considering an extended trip to Asia. Mainly Japan, Nepal, India, and Tibet. I have never visited this part of the world and have always wanted to so this will be a major tick off the bucket list!

 

What would you do for a career if you weren’t doing this?

If I wasn’t working as a developer, I think would like to be drum teacher. I have played drums since I was very young and have earned a Grade 5 distinction for playing. I would love the time to commit to earn my Grade 8 to become a certified teacher. I have a major passion for music and love sharing my knowledge and passion with others.

 

What do you enjoy most about working at IMGS?

The thing I enjoy most about working at IMGS is the learning experience. IMGS has such a competent and professional team that I feel privileged to be able to work alongside and learn from. I have learned so much since arriving in the areas of database design, location intelligence and project management and it is all down to the help provided to me from each of my colleagues. There are no stupid questions here, which I’m happy about as I ask a lot!

 

Pragruthi Morali

Women in tech at IMGS – Pragruthi Murali

Women in Tech at IMGS - Pragruthi Murali

To celebrate Women In Tech at IMGS and their invaluable contribution to the business, we will be turning the spotlight on key female members of our tech team over the coming weeks. 

This week’s Women in Tech at IMGS spotlight interview is with Pragruthi Murali, Application Engineer, IMGS.

Pragruthi Murali, Application Engineer, IMGS

I’m an enthusiastic computer science engineer with an MS in Software Design and Development . I’ve been working with IMGS for the last 3 years as an Application Engineer.  My work revolves around development, implementation, data engineering and dba tasks.  I work with customers to provide technical support for geospatial applications which include Geomedia, M.App, G-tech managed service and I’m currently working on adding FME to my list!

Pragruthi Morali

Pragruthi what first got you interested in Tech?

As a kid I had a fascination with a PC game ‘Ages of Empire’. It still fascinates me today how clever the in-game AI was and how much tech has evolved since then.

In middle school once of my subject choices was computer science, which introduced me to the world of tech at a very early stage.

 

Can you tell us a little about your role and what a typical day is like for you?

Recently no two days have been the same. I have been fortunate to get exposed to various aspects and stages of projects from design  development, implementation and support to drafting estimates.  I love the problem-solving aspect of my job: resolving bugs, developing new product ideas and experimenting with new features, which all help to keep me engaged and busy.

 

What obstacles do Women in Tech face?

Irish statistics show that less than 25% of women work in STEM skilled roles.  This gives an impression that tech might not be a great choice for women, which is absolutely not the case.  As Ireland is now widely viewed as a leading technology hub, more needs to be done at school level to educate girls about the the vast array of opportunities that are available in the STEM World.

 

What advice would you give to a woman thinking of pursuing a career in technology? 

If you are passionate about technology, then go for it as there are a myriad of opportunities for women in this field. It’s a hugely rewarding career with plenty of scope for career development.

 

What are your thoughts on the next transformation in the tech industry?

Post-pandemic there may be an online learning revolution for students, which increases the demand for LMS (Learning Management System). I previously used Moodle cloud for one of my thesis on LMS and would highly recommend it.

Also with bitcoin and blockchain becoming popular terms it might be good to see quantum computer and quantum key becoming a reality outside of the lab.

 

Are there are particular women in tech that have inspired you?

It was inspirational to see how Sheryl Sandberg, Chief Operating Officer at Facebook, led the social media giant from a $56 million loss in 2008 to an astounding $18.5 billion profit in 2019.  I was given Sheryl Sandberg’s book ‘ Lean In’ as a graduation present, which is a great guide for job hunting and finding your way early in your career.

A quote to live your life by: ” If your offered a seat on a rocket ship , Don’t ask what seat!  Just get on “- Sheryl Sandberg.

 

When travel restrictions are eventually lifted what destination would you most like to visit?

I would love to visit Greece, in particular the beautiful island of Santorini with it’s picture post card scenery . I also  can’t wait to try their famous classic baklava!

Women in tech at IMGS - Aoibhinn Stuart, Delivery Director

Women in tech at IMGS – Aoibhinn Stuart

To celebrate Women In Tech at IMGS and their invaluable contribution to the business, we will be turning the spotlight on key female members of our tech team over the coming weeks.  This week’s Women in Tech spotlight interview is with Aoibhinn Stuart, Delivery Director, IMGS. 

Aoibhinn has over 15 years project delivery experience across a wide variety of industries, including Banking, Payments, Insurance and Utilities.  She is passionate about customer service and the importance of building strong collaborative business relationships. She is ISEB, ITIL and Scrum.Org accredited with a strong understanding of complex IT systems and processes.  Aoibhinn specialises in project management, process improvement and quality assurance to improve system stability, functionality and efficiency.

Women in tech at IMGS - Aoibhinn Stuart, Delivery Director

Women in Tech Spotlight Interview with Aoibhinn Stuart.

Aoibhinn what first got you interested in Tech?

It’s hard not to have an interest when everyone in your family is involved in tech in some way!  I remember desperately trying to get a hold of my brothers Atari and Commodore 64 when I was young and later in school we were introduced to the concept of coding, which really interested me.  I always had a love of engineering and tech and so it just felt like the right route for me.  It certainly helped to have been encouraged from a young age and exposed to the world of tech, long before I began working in this field.

Can you tell us a little about your role and what a typical day is like for you?

My official title is Delivery Director but in reality I am part-time PM, part-time tech lead, sometimes strategist and all day problem solver! 

My mornings consist of catching up on e-mails, chasing my children around to get dressed and doing the school run.  Most mornings we have a stand up to catch up with everyone virtually and go through the tasks of the day.  Usually this is followed by some project calls with customers and in the afternoons I will either have a new project plan, estimation or a marketing strategy to review. 

Since the pandemic, I have made a huge effort to regularly link in with each member of our team individually for a ‘virtual coffee’. It is really important to reach out and ensure we look after one another as these times are very isolating and as leaders we must lead by example.  That means making time for people and checking in with them on a personal level.  It’s too easy to lose sight of the human element of business when you are caught up in the deadlines and tasks of everyday life. 

Come the evenings.  It’s homework followed by dinner then a walk and usually a movie.

I do a final e-mail catch up before bed and check my calendar so I am prepared for the next day.

What obstacles do Women in Tech face?

Themselves.  When I speak to women in other industries or younger women deciding on a career choice, I often hear that they don’t like coding – therefore they wouldn’t like tech.  This common misconception drives many women away from our industry. Tech has such a vast array of jobs that I am still amazed at some of the roles people enjoy!  Coding is a small part of what we do.  There are business analyst roles, project management, solution architects, data driven solutioning, the list is endless.  If I could share my experience of tech it would be that women are great leaders, enablers, multi-taskers, mentors and yes, coders! There is room for everyone!

What advice would you give to a woman thinking of pursuing a career in technology?

Go for it!  It really is a fantastic industry to be  part of.  It’s never boring, things change so quickly that you are kept on your toes. Do your homework, look at all aspects and decide which interests you the most.  Is it infrastructure, data, application, web development?  Speak to others in those areas to get a feel for what their role entails.  Encourage each other.  There is an expression I love ‘real queens fix each others crowns’ and that would be my mantra for women in all industries.

What are most proud of in your career so far?

The biggest success for me is that I have managed to juggle a career with bringing up a family.  It is not easy to manage the pressures of young children and a busy job but somehow I have just about managed to keep all the balls in the air!  Joining IMGS has been a real career highlight for me, we have such a fantastic team and I love my job and who I work with.  I don’t think you can wish for more than that in any role.

What do you like to do in your spare time?

Pre COVID, travelling would have been my big love.  I am really looking forward to getting back to visiting new countries and seeing new cultures.  Nowadays we do a lot of walking, cycling and movies!

When travel restrictions are eventually lifted, what destination would you most like to visit?

A bar.

Moving to the Cloud by Ciaran Kirk, Operations Director, IMGS

In today’s changing world, work practices have changed forever, and on-premise systems are proving expensive to maintain while not providing the level of access and performance needed.

Moving to the cloud is not a new concept, and where previously it was a nice tech option, it is very quickly becoming the only solution available to organisations, who want to:
1. Decrease infrastructure capital and operating expenditure
2. Provide true resilience for continuity and disaster recovery purposes
3. Provide elastic scale to handle peaks in demand
4. Query large datasets which have outgrown on premise data storage
5. Take advantage of the latest AI and Machine Language analytics to gain rapid insight

If we take each one of these points in turn, we can see why moving to the cloud is a key part of any digital transformation:

Decrease infrastructure capital and operating expenditure
By moving your systems to the cloud, you are removing the need to have on premise servers thereby decreasing your systems administration overhead. This can be a significant cost saving, if done correctly and at the right time.

One option is to look at IAAS – Infrastructure as a Service – where you migrate your servers to cloud providers such as Amazon EC2 or Microsoft Azure. In this case the cloud provider is providing the infrastructure and will ensure your servers are physically up and running.

Moving to the Cloud Blog Picture 2         Moving to the Cloud Blog Picture 3 final

In the case of IAAS, your IT support team must still maintain the system running on the server, so it doesn’t completely remove the need for IT systems support.

If the aim is to remove or greatly decrease systems administration overhead as well as the infrastructure costs, then SAAS (Software as a Service) is the perfect model. With the SAAS model the supplier handles the infrastructure and the software system and support so that the end user has little or no overhead in running the system.

In IMGS we have long been users of Office 365, and as someone who remembers when we used to have the headache of running our own exchange servers, Office 365 has been fantastic and provides access anywhere and is always available.

One of the big advantages of SAAS solutions, is the pay as you use model. With this model, customers don’t have to make a large capital expenditure up front for a software solution. This makes it easier in future to switch solutions if required and avoid a legacy IT trap where customers can’t afford to switch off platforms.

In IMGS again we have experience of this, when recently we moved from Salesforce to Zoho. We were able to switch seamlessly and migrate from one system to other in 2 days with no downtime or outages – if these solutions were on premise this would have been a lot more expensive to do.

Provide true resilience for continuity and disaster recovery purposes
One huge advantage for moving to the cloud is in providing true resilience and disaster recovery options. With cloud systems, you can select to have cloned systems in different regions to make sure that if there is a major outage, your organisation can continue to operate.

By selecting high availability options, you can ensure your users are never offline no matter where they are in the world. With the increase in remote working, having systems that are not dependent on a single site or broadband connection is critical.

Provide elastic scale to handle peaks in demand
The real power of the cloud is the ability of cloud systems to scale quickly to meet increased demand. With on premise systems providing scalability can be very costly, as hardware needs to be purchased in advance and will sit idle when demand is low. With cloud you can scale up and only pay for what is needed.

With Serverless technology, cloud providers are bringing this scalability and availability to a new paradigm. Serverless allows organisations to build and run high availability applications and services, without worrying about provisioning, maintaining, and administering servers.

Query large datasets which have outgrown on premise data storage
A big driver to the cloud is the explosion of data growth. Putting it simply, large organisations do not have the capacity to store all this Big Data on premise and the cloud is now providing the only real option. Cloud providers can provide a wide variety of ways to store your data whether in low cost file storage such as Amazon S3 to data warehouses like Snowflake.

A big concern for users is Data Residency and where this data is stored. In Ireland we are very lucky with most of the cloud providers providing data centers in Ireland so we can ensure our data stays within Ireland and the EU to meet GDPR regulations. Also, with modern integration tools like Talend we can also take a hybrid approach and store sensitive data on premise and in a variety of best of breed cloud locations, so as not impact end users.

Take advantage of the latest AI and Machine Language analytics to gain rapid insight
The final reason to move to the cloud is to take advantage of the powerful machine learning and AI services that are available as cloud services and can deliver rapid insights in areas such as predictive maintenance and customer churn.

All the major cloud providers provide these capabilities, (for example Amazon SageMaker) as well specialist SAAS providers such as Databricks provide data science and analysis capabilities that on-premise systems just can’t match.

Moving to the Cloud Blog Picture 4

Summary
As I said at the beginning the cloud is not new and a lot of what I have discussed here is pretty obvious, but I do feel organisations haven’t fully embraced the cloud and I hope this blog might help your organisation begin their digital transformation by moving to the cloud.

If you would like to learn more about moving to the cloud please attend our upcoming moving to the cloud webinar where we will demonstrate using our Data Intelligence Platform how you can migrate to and take full advantage of the cloud:

Webinar: Migrate to the Cloud with IMGS

FME 2020 by Gavin Park, Lead Solutions Consultant IMGS

It’s likely that 2020 will be remembered for a long time so it is perhaps fitting that the focus on the FME 2020 release has been finding your data zen and making working with data even easier.

FME started 2020 ready to prove it had come of age and was no longer that geeky tool sat on the GIS computer, but was a tool that delivered across the whole organisation. 2020 has proven to be the year that has shown it can.

FME is for the Organisation

FME’s goal is to support individuals and organisations of any size with integrating departments, applications and systems. This is a goal that has become increasingly important, as organisations try to adapt to new ways of working and increased demands for data to meet new challenges.

Sisense Bootcamp in Israel by Ellen Noone

On day one of the Sisense bootcamp in Tel Aviv Israel, myself and Ciaran arrived SisenseBuilding1at the Atrium Tower to start our bootcamp. We were immediately wowed by the impressive skyscrapers and towers surrounding us. The building hosting us, had 40 floors and we were on floor 16. We were met by Einat Pick, the organiser of the bootcamp. We had a very welcoming reception where we were given goodie bags and the opportunity to speak to other Sisense partners, who were also attending the bootcamp from all around the world. …