Still Living In A Flat File Interface World

Posted by: on Apr 20, 2010 | 14 Comments

Guess what?  The rumor on the street is that HR leads the industry in using the Software As A Service (SaaS) delivery model.  Surveys say that customers still prefer best-of-breed solutions over integrated solutions so that means more HR applications instead of less.  Guess what also is true about many of these SaaS applications?  They still use flat files as the main vehicle to shuttle information into and out of the application. In this model the customer is still required to develop an interface to the SaaS vendor.  Wait, what?  As a customer you still have to do custom development even though you are using a SaaS application.

There are several other more state of the art options such as an Enterprise Service Bus (ESB) or SOAP messaging.  I don’t know about you but when an employee gets hired into my core HR system, I would love to have the employee information zip over to my SaaS based Learning Management System within a couple seconds instead of sometime that evening when the flat file interface gets run.  When I push the button to fill a requisition in my recruiting application, wouldn’t it be great to have the new hire in my core HR system right away. Sorry SaaS Recruiting vendor, sending me XML files via FTP multiple times a day doesn’t count.

All is not lost fortunately.  The HR SaaS application vendors are starting to incorporate messaging or ESBs into their offering.  The most prominent example of this is Workday purchasing ESB vendor Cape Clear back in February of 2008.  Since then Workday has embedded Cape Clear into their application to allow end users to create their own real-time inbound and outbound interfaces.

For those other SaaS vendors who are still stuck in the flat file interface, you better get it together soon.  We are wising up and will be requiring state of the art real-time interfaces in our vendor selection criteria.

14 Comments

  1. Trish McFarlane
    April 21, 2010

    As my little boy would say, “THIS is what I’m talkin’ about!” I have to admit, having this be the first thing I’m reading after waking up my head is spinning. I already see a couple of the links I need to click on to learn more about what it is and what it all means.

    Thanks for helping us in the day-to-day HR learn HOW these software options work. This is the exact type of thing I hope to learn more about at HRevolution. Thanks Mike!

    Reply
    • Michael Krupa
      April 21, 2010

      Trish – I recommend moving my blog to the third thing you read in the morning. I don’t want to get hit with a Workers Comp bill when you hurt yourself because your head is spinning.

      Reply
  2. Ian Alexander
    April 21, 2010

    Hi Mike. Very enlightening post on interfaces. Lest everyone think vendors aren’t wise to data interfaces, allow me give you a vendor perspective.

    At Cytiva / SonicRecruit, we can do very easy service oriented data interfaces that are real time. However, they are not without some cost and some investment of resources from the client. Nineteen out twenty times, clients will opt to push and pull flat files because there is no cost to it and they can live with the results. The client can maintain it through our robust built in connection tools.

    SaaS means, to many purchasers, “I don’t have to include my IT resources on this,” so it’s not surprising that they choose a cheaper, simpler method and sacrifice a couple hours for data transfer.

    That said, our typical client is a couple thousand employees, not a couple hundred thousand, so my perspective is based on a mid market sized company.

    I see how, from your perspective, you see vendors pawning off less than perfect solutions. From my perspective, clients are opting for less than perfect solutions.

    Reply
    • Michael Krupa
      April 21, 2010

      Ian – Good point. I think if more vendors offered built-in and easily configurable web services interfaces for their products then customers could connect them without having to request flat files. I agree that SMB customers may not have the ability to work with SOAP and ESB based interfaces.

      Reply
  3. Shannon Whitley
    April 21, 2010

    Mike brings up some great points. A vendor should at least offer more advanced data integration tools. On-the-other-hand, a lot depends on the needs of the client and the situation. I recently met with a vendor who laughed because I *wanted* to use a flat file. There are many cases where it’s more efficient to implement a simple file transfer. Although I think Mike’s point is that multiple options should be available, I’d also like to avoid labeling flat files as undesirable just because they represent older technology.

    Reply
    • Michael Krupa
      April 21, 2010

      Shann0n – Thanks for pointing this out. The vendors should be offering both basic and advanced integration tools. You are correct that there are times when a flat file interface makes sense. I didn’t mean to imply that flat files were undesirable, only that it’s time that vendors also start offering more advanced interfaces. Thanks for stopping by.

      Reply
  4. Beth N. Carvin, Nobscot Corporation
    April 21, 2010

    Good points made by all.

    When Nobscot Corp. first started offering “data feeds” we gave the choice of a once a day file transfer or an “upon the event” xml file via http request . Almost all clients chose the former even though the latter might be easier and more timely. They were familiar and comfortable with exporting a file and sending it via scheduled ftp.

    I think it’s probably been about 8 years now and we still see zero demand for anything other than data file via secure ftp. I can understand where they are coming from. It’s a pretty simple process that works well for most applications. I’d like to see companies use some of the newer methods for data transfer but if they are happy and it works well for them that’s fine too.

    Reply
    • Michael Krupa
      April 22, 2010

      Beth – I know, I know. I can’t cut customers any slack in this either. FTP + Flat Files work well and as Shannon points out there are legitimate reason why you might want to use flat files. However, as we continue to roll out Manager and Employee Self Service in our core HR applications and the outlying best of breed applications, the users are not going to put up with 24 hour old information forever.

      I’m glad you offer an event based XML feed via HTTP. When you customers are ready for real-time feeds you will be one step ahead.

      Reply
  5. Scott Simpson
    April 22, 2010

    As a provider of HR SaaS solutions I have to be another voice saying the demand doesn’t seem to be there. I would love to have all our clients connect dynamically via web services or similar, but most multinational organizations in my experience aren’t ready to do that. HR are generally our clients, and due to resource issues, there’s limited availability of internal IT staff to do any data mapping activities.

    There are also major security standards issues. Organizations have a clean security approved process for doing file transfers, and any new data transfer process requires a whole security approval.

    One hope for the future may be with providers of SaaS Data transfer/Integration services. There are a few of them out there now. Rather than an organization integrating with all their providers, they integrate with a SaaS data integration service. Other SaaS vendors build connectors to/ from the SaaS data integration vendor, and that gives seamless data transfer with a whole variety of SaaS vendors.

    That whole process becomes a lot easier when the different vendors implement a common data standard though – I’m not sure how widely the HR XML Consortium standards are implemented in vendors products. It will be interesting to see how this whole data integration SaaS segment shakes out over the next few years.

    Reply
    • Michael Krupa
      April 26, 2010

      Scott – Thanks for your input. I guess I am a lone voice out there looking for more real-time interfaces. I think the lack of mature tools and data exchange standards are a factor in the slow adoption by customers. I have not heard much recently on HR XML. It’s on my list to do some research and see what is the latest on this standard.

      Reply
  6. Meg Bear
    April 23, 2010

    Mike,
    Glad to see you bring up the topic. Having studied the challenges of integration from all sides (seriously, check my resume ;-) I can tell you first hand that I have built slick and fancy solutions in the past, only to have people ask for flat files. The key is to make it easy for customers to use and oftentimes that’s how flat file solutions are born.

    I think it’s a big help having people like you explain the value of a message-based solution. I think that’s the part that’s been missing so far, the *why*. I don’t really think this is a technical problem, and I do have hope for standards but then again I’m a tech nerd and a dreamer.

    great post!
    -Meg

    Reply
    • Michael Krupa
      April 26, 2010

      Meg – I think it is a technical problem from the standpoint that the tools are still a little cumbersome to use although they are getting better quickly. I find it fascinating that many HR departments seem frustrated when the data interfaces between core HR and best of breed systems are not real-time because nightly flat files are used to shuttle data around. Yet at the same time due to the reasons given by other people in their comments to this post (cost, security, unfamiliar with the technology) real-time interfaces are not developed. Standards will definitely help. Thanks for your comments.

      Reply
  7. Deirdre
    April 23, 2010

    You are so smart it scares me. Why isn’t everyone like you?

    Reply
    • Michael Krupa
      April 26, 2010

      Oh dear, you definitely don’t want everyone like me, but thanks for the compliment.

      Reply

Leave a Reply