Home > Configuration, OpenERP v6.0, Ubuntu > ETL and OpenERP

ETL and OpenERP

Figure 1: Simple schematic for a data warehous...

Image via Wikipedia

ETL stands for Extraction, Transformation and Loading. They are used to integrate different systems in different platforms, and they do it in batch mode, they don’t do it in real-time (well, there are some tools that do it, but most of the times we integrate the systems on a periodic basis).

I will not explain how an ETL system works, I will explain why you should use it with OpenERP. In a latter post I will cover the topic of updating OpenERP data with ETL.

Why use ETL? Many reasons. The first reason is it is a very productive tool and once you learn how to use it, your productivity as a developer increases twofold (at least). Developing integration jobs is quite intuitive and they allow you integrate diverse environments, such as data from different databases and operating systems in a single job.

The second reason is these tools already have integrated a production environment for monitoring and executing the integration jobs. These environments have grown more robust along the times and its logging facilities are quite impressive. OK, you can do develop the environment yourself, but it takes time and testing, it is not cheap.

The third reason is, you will always need to integrate systems with OpenERP. Always. As customers get larger, they tend to have more systems in place. That’s the law in this industry. And you can not change them, you have to live with them. And living with them means, integration your OpenERP information with their information. And their information with your OpenERP information.

The last reason is, ETL applications save you many hours when it comes to migrating data into your system. MANY HOURS.

Now, when it comes to OpenERP and ETL, you have some options. I just looked at two of them. Talend and Pentaho Kettle. These days we are starting a project that needs to extract information from an accounting system and integrate it with the MRP module in OpenERP. I think we will end up using Kettle, because it already has plug-ins for loading data into OpenERP. Have not seen that yet in Talend. I will cover those modules in a coming post.

Cheers!

Advertisements
  1. February 9, 2011 at 11:44 am

    Hello, nice blog. Still, I would like to mention we already have a killer tool for OpenERP + ETL: we have a Kettle ETL (Pentaho) plugin that is able to communicate with OpenERP and use it’s full API (so enforce access rights, workflow, business methodes etc…), it’s called TerminatOOOR, take a look at the doc and the few demos:
    https://github.com/rvalyi/terminatooor
    We will soon document it better as lot’s of improvements are coming…

    • developingmoldeo
      February 9, 2011 at 1:03 pm

      Thanks! Actually I heard many good things about TerminatOOOR (cool name BTW), so I was planning on testing it next week. I will check the documentation and in case I have any question, I will e-mail you
      Thanks again!

  2. February 14, 2011 at 7:49 am

    Nice Article. I wanted to mention that we have used Talend very effectively for OpenERP Data Migration. I think you should explore Talend. It has a learning curve but it is very powerful. Another direction Talend could be used is for Data Integration when you are trying to connect 2 different applications. We have successfully connected OpenERP with Magento without using any connectors. It is purely database level mapping without need of writing code which I think is bit difficult to maintain between different versions.

    Cheers,
    Shabbir.
    Pragmatic
    http://www.pragtech.co.in

    • developingmoldeo
      February 14, 2011 at 2:43 pm

      Thanks Shabbir!!!! I hope to find the time to try Talend, might do that next week, in case I do I will post an article with my experience. BTW, I wanted to congratulate you for the OpenERP Video website, it is really cool and saved me a lot of time! Actually, I will post on this website now.
      Cheers,

      Gustavo

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: