Importing: Best Practices
A Helpful Guide to Data Building and Management: Part Two
2 August, 2021 by
Importing: Best Practices
Micah Methvin (DO NOT USE)
 

Where to Start

Are you struggling with your databuild in Odoo? It’s a common point of stress during implementation, especially when switching from a legacy program that may not have direct parallels to Odoo’s user interface and record relationships.

Here at Hibou, we’ve seen data of all stripes, and there is usually a quirk or two about each of our clients that makes getting started the hardest part. We’ve been at this long enough to have a few recommendations to help you get your database up and running. 

Essentially:

  1. Start with clean data. 

  2. Make a plan, and then test it.

  3. Build a foundation before building relationships.

  4. Take your time. 

Clean Data is Key

Moving away from an older system is about as much fun as cleaning out the garage. You know what’s in there, you also know you haven’t looked at it with a critical eye in years. The impulse to hoard old records is strong, but let’s face it, if you don’t use it every day, you can likely live without.



Now is the time to trim down and reassess what information still has a valid use-case in your company’s day-to-day workflows. Do you really need those deprecated product templates? When was the last time you had a meaningful conversation with John Smith about his sale eight year ago? Not to mention the years of clutter from typos, duplication, abandoned projects, internal failure to research, old marketing strategies...the list goes on. 


This can be the most tedious part of your migration to Odoo, but in our experience, the first step to a usable new database is scrubbing the old one out. Take the time to talk to your other managers, department heads, and employees. Their feedback can be critical to determining what information they cannot live without, and what portion of the existing data is still relevant.

Many of our clients will only import contacts and sales leads from the last three years, for example. If you’ve ever wanted to reorganize your product categories, fix an annoying naming convention, or revise an internal workflow, now is the time. 

Enter:  


For those of you starting in v14, you’re in luck! Odoo launched a new application called Data Cleaning that can assist with finding and correcting inconsistencies once your data has been loaded. It’s worthwhile to spend some time refining your data before import. Spot check, spell check, clean up punctuation and trailing white spaces, and you’ll be better positioned to make immediate improvements. Most database services offer an export tool of some kind, so get familiar with that function and the kind of document it produces.


Make a Plan!

Once you’ve reviewed your data, it’s time to begin comparing it to Odoo’s stock fields. Hibou can help you with this assessment, adding additional storage fields for key information. Tailoring the database to suit your needs is a big-picture strategy, however, and the eventual application of the information should be kept in mind. Working through how the data will be used goes hand-in-hand with early workflow development, and should guide customization. 




Hibou recommends starting with a sample mix of your data and entering that manually into your test environment. This will allow you to step through the stock functions to hone in on functionality gaps that need to be addressed. It will also help you understand how certain fields are used within the database, and their relationship to each other.


Once you have a plan, test it! There is no worse feeling than getting an import put together for thousands of rows, pulling the trigger, and then realizing you’ll need to go back and reclassify, edit, or change configuration. There are methods to do this, covered in a previous export article, and realistically, you should expect a few revisions along the way. Having a plan can minimize this back and forth and provide an excellent roadmap not just to the data itself, but how it’s used.


One other common pitfall Hibou has experienced is over-building the staging or test environment. It is extremely convenient to have a sandbox environment where there are no real consequences for testing and training, but resist the urge to make it resemble the end product too closely. Production data ultimately belongs in production, and test environments can be great for testing import templates, but moving that data from test to production via script is a lengthy, expensive, and imperfect process. Once you have a working template that checks all the necessary boxes, test on test, load into production. Don’t pull the trigger on an import template without first achieving that blue banner!

Build a Foundation, Then Relationships

In the previous article about exports, we discussed flat vs. relational data. In general, flat data is information that is entered and easily editable through the user interface. Relational data reflects the relationships between information in your database, often between other applications such as Sales and Inventory, or Employees and Payroll. 


When viewing a sample record in Odoo, there may be required fields. These are subtle, but the entry line will be bolded with a thicker black line. When creating an import template, the bold fields must contain data to allow creation. 



Flat data will be presented as an editable field, where relational data is more often a selection that converts to a link upon saving. The key to establishing a relationship between two data points? Making sure they both exist. Layering data in stages and then linking it together with a third update import is the cleanest way to build a path through data that is used in multiple places. For instance, in order to import your product list pre-sorted into their nested categories, the category structure must first exist. 



In some cases, in debug mode, you can create missing options directly from the import wizard by using the “Create if doesn’t exist” check box on the header row. This is useful, but isn’t available on all fields or record types, and it can be prone to error when working with larger data sets. That ‘new’ item could simply be a typo in your spreadsheet.


A typical import sheet for Hibou contains at least three pages and involves a merge tool and some formula work. We first create the category template with all related configuration points, and import it. Then we clean, organize, and map our product export template from the legacy program to match field names in Odoo (use debug to pinpoint the exact field name you need for your header). We can then export our existing categories with their external IDs, add this to our template as a second page, and then merge/match categories into the product import for each item. 


Steppy? Sure, but often better than manually combing through every product template to set the category manually, or using a server action to search, filter, and assign one category at a time. With a little patience and attention to detail, you can build relational data into your import templates and save several hours of labor.


Take Your Time

We are often working under deadlines when trying to implement a new system, however, there is an old adage about measuring twice and cutting once. Do you want Odoo fast, or do you want it right?

Spending a little extra time at this stage can save many hours of revisions and effort down the road, so slow down, take a breath, and work with intent. Plans evolve along with workflows, so work closely with your implementation partner and your internal team to keep the big picture in mind. It is not at all uncommon for databuild to take three months or more, especially when migrating from older systems. 


Experiment with the options in test, and take only good data into your production environment. If you work with patience and intent, you are much more likely to end up with a database you recognize and can navigate afterwards.

Importing: Best Practices
Micah Methvin (DO NOT USE) 2 August, 2021
Share this post
Tags
Archive
Sign in to leave a comment