So we are almost ready to go.  In this data migration series, we have already looked at how to migrate data and some tools to help with the migration.  But now for the fun part, I want to share with you some data migration tips to help make your migration into Salesforce as smooth as possible.

Let’s jump right on in, and please share any of your own tips in the comments section below.

Data Migration Tip #1: External IDs & Upserts

Let’s walk through a common scenario in a data migration…

The Data Migration nightmare:

You are about to migrate 12 different objects into Salesforce, everything from Accounts, Contacts and Tasks through to Opportunities, Cases and a number of custom objects you have created in your org.

How do you create these records in Salesforce, and still have all related records end up linked together?

Do you insert the first object (let’s say Accounts).  Because you need to link the Contacts to those Accounts, in the second file do you use Excel to create the link?  Do you take the import file from Dataloader, run a VLOOKUP in the Contacts file and then pull in the related Account IDs?  YUCK!  Now imagine having to repeat that process for all 12 objects…

I have been there myself.  And when it is a very large migration it can become extremely difficult to track and manage all the spreadsheets.  Especially when you are working with live data, that is always changing and is never static.

This scenario can be the nightmare of any data migration but it doesn’t need to be.  #NoMoreVLOOKUPS

Salesforce External IDs:

As you know, each Salesforce record is given a unique 15/18 character ID.  That is great, if you are only dealing with data in a single system or Salesforce org.

But what happens when you are need to store the ID of a record, that is actually sourced from a system outside of your Salesforce?  For example an ID from an invoicing system like SAP or Oracle.

This is where External IDs come into the picture.

By configuring a field to be an External ID you can then use it to store the original reference/ID from the other system.  Additionally, you can also use it to prevent duplication within Salesforce by setting the External ID field to unique.

It is this ‘uniqueness’ that helps us out when migrating bulk data into Salesforce.  As Salesforce will automatically index the External IDs, allowing you to use this as a reference when doing an Upsert within Dataloader.

#NoMoreVLOOKUPS: Upserts with External IDs:

Now let’s take a quick look at Upsert within Dataloader.  Upsert combines the ability to either Insert/Update records while using a single file.  However, using Upsert in Dataloader also gives you access to a few extra features.

After you click Upsert and start going through the Import Wizard, you will get the ability to select either the Salesforce ID or an External ID (if available on the object).

Additionally, for any relationship you have on the object, Dataloader will also prompt you to select the applicable ID fields which is also referenced in our import file.

This is how you avoid using VLOOKUPs!  

By populating an External ID, on each record you load into Salesforce you can then use this to get Salesforce to do the lookup for you.  Even if you don’t actually have an External ID from another system, you can create your own.  Just make sure it is unique for each record.

I know this is a little confronting to start with.  It took me a little while to wrap my head around just how exactly to do this when I ran my first data migration.  However Upsert & External IDs can save you so much time, I strongly recommend you practice and learn how to use this ‘double act’ to your advantage 🙂

There is of course more resources available on this topic.  Salesforce Help has an article which gives you a step by step process you can follow.  I also found a video series on YouTube by Doug Ayers, which really breaks this down over an informative video tutorial.

Data Migration Tip #2: Update System Audit Fields (Salesforce’s ‘hidden’ trick)

In Salesforce, every record has a number of system date fields which typically can’t be edited.  Fields like ‘Created Date’, ‘Created By’, ‘Last Modified Date’ and ‘Last Modified By’ are locked down (and for good reason!)

These are your standard audit fields and are very important when tracking a record’s history.  While planning a data migration, you might come to realise that you need to retain this information from the previous data source.  And that is where the permission ‘Update System Audit Fields’ comes into the story.

Imagine a sales person’s activity report, which is displays only activities created in this quarter.  As such after a data migration is complete, you still want your users to run these reports.  So what can you do?!

In the past, being able to edit these fields was part of what I called the ‘hidden Salesforce toolkit’.  You know the list of things that Salesforce can do, but they don’t really publicise? Like increasing the cheeky workflow limit here or there…  But all part of the Salesforce Support toolkit, but you had to ask for it to be enabled in your org first.  The problem was, you had to know them in the first place you could even ask, so that you could get it requested to be turned on!

Winter ’16 Release & Update System Audit Fields

In Winter ’16 Salesforce made a number of these ‘hidden’ features publicly available.  And with a few clicks you can now enable it easily for your data migration.

There are a few considerations to look at before you enable the ability to update the System Audit Fields, so check them out here.

And then head over to here, for instructions on how to setup and enable this feature in your org!

The biggest point I will highlight about accessing these fields, is that they are only available on record creation via Dataloader.  Meaning you can’t insert a record and then try to update the created date on it separately.  If you did try to edit/update these fields after a record has been created you will get an error.

Data Migration Tip #3: Automation Anchors

Automation with Salesforce is fantastic.  Admins and Developers have access to a wide array of automation tools within most Salesforce orgs, including Process Builder, Triggers, Workflows, Sharing Rules.  But when you are migrating data, they can quickly become an anchor.

What do I mean by that?  Do you know what automation will fire off based on the data work you are doing?  For example, is there a workflow that will send an email alert to another team, or even the customer directly?  Is this what you expect to happen, if so that is OK.  But you do need to make sure you know what any data you load or update into the system will do to the existing automated processes in your org.

In addition, automation in Salesforce can end up adding extra time to any data insert, update or upserts.  Remember even deletion of records can fire off a trigger!  When you are moving or manipulating a bulk volume of records, this time can easily add up.  Complex sharing rules might really slow down an insert/update of records.  And depending on the volume of data you are working with, you of may end up hitting some of Salesforce processing limits.

So before doing any data migration, test and make sure the automation you expect to work does.  Otherwise see if you can deactivate the workflow while you are doing the migration.

Share your tips

This is by no means a definitive list, and there is plenty more to come.  But in the meantime, please share your own experiences or tips in the comments section below!

Happy migrating!

Everyone loves data!  In my last post I shared with you was some thoughts on how to make sure a data migration goes smoothly.  There is a treasure trove of tools out there that can help you with any Salesforce Data Migration (more than I can put in this single post!).

In this post, I am going to share some of the most popular tools to help you out with any Salesforce Data Migration, and best of all they are all free!

Tools to help you create & edit records in bulk

Salesforce has a good data import wizard which can guide you through basic record inserts, like when you need to create a bunch of new accounts/contacts.  But what happens when you need more power?

First thing is first, the tools below will respect any object/admin permissions you have within Salesforce org.  Meaning if you don’t have access to create records on a certain object, you won’t be able to create records via one of these tools.

1) Dataloader

This is a tool developed by Salesforce themselves.  Once installed on your PC, you have a powerful tool to help you to migrate or update data in Salesforce.  The interface itself is a bit basic, but it is relatively straightforward to use and allows you to either Insert, Update, Upsert, Delete and Export records from any object in Salesforce that is available via the API.

Most of the features of Dataloader are available through a simple user interface, but if you want to unleash even more power there is also an advanced option allowing you to use a command line interface.

It is worth mentioning that Apex Dataloader only supports .CSV files.  So if you have Excel/Google Docs, make sure you save the file as a .CSV file before starting your Salesforce data migration!

Final thing to point out, this tool uses SOAP API, which means you need to be on Enterprise, Unlimited or Performance Editions of Salesforce.

To get it, once logged into Salesforce, go to Setup -> Data Management -> Data Loader.   Once there you have options to download the installation file for Windows or Mac.

To read more about Dataloader, click here.

2) Jitterbit Data Loader

This is a tool is similar to Apex Dataloader in most regards.  The basics are relatively similar – you can still Insert, Update, Upsert, Delete records though has a few extra tricks up its sleeve!

First up Jitterbit’s tool supports more versions of Salesforce, with Group Edition the minimum requirement!  Meaning many more people can use this tool versus Apex Dataloader.

Secondly, this tool supports more than just CSV files, it works with any flat file or even a database connection for the more advanced users out there.

You can also perform basic transformations with your data on the fly, and link your configuration settings to the Jitterbit Cloud.

It also has a relatively simple point and click interface with a helpful wizard to guide you through the steps.  Where to get it?  Head on over to Jitterbit’s website to register, download and install.

3) Field Trip

In my last post, I made reference to making sure you understand the data you are migrating.  Question, analyse, question and repeat!

Field Trip is a free tool available on the AppExchange, which helps you understand the data completion rates already in Salesforce.

Why is this helpful? You can view which fields actually contain data, represented as a percentage across the entire data set within an object.  This is very handy if you are looking at removing technical debt within your org, or trying to understand what fields are most used.

Imagine you are reporting on the Account table.  A field that has 100% completion rate means every single record has a value in that field.  This enables you to then analyse the fields that are most used and ignore the fields which aren’t.

Go forth and migrate!

We are only just skimming the surface here, and there are plenty of other tools you can use for a Salesforce data migration.  As an example, Salesforce also have Workbench which is an web-based version of Dataloader, with a few differences.

Everyone has their own favourites, so please feel free to share yours in the comments below!

In my next few posts, we will cover some popular tips to use when migrating data.

There comes a time in every administrator’s life, where you might have to migrate data into Salesforce.  Why is that important?  Data is the core of any CRM platform and has the power to become a blocker for your customers and/or internal users!

A request to migrate data could be as simple as someone asking you to import data into Salesforce from a spreadsheet.  Or it could be due to the retirement of another system.  Even a merger/acquisition!  But when the time comes, you will need to be able to really get under the hood to understand the data and create a data migration plan.  Because without a plan, migrating data can become a massive risk to any project.

Over the next few weeks, I am going to dive into a number of topics relating to all things data migration.  YAY! I hear you screaming! Also please feel free to share any of your experiences in the comments below, as with all things data migration there is always plenty to learn!

So let’s get started and dive into the key things to plan for when you are attempting to migrate data into Salesforce.

Everyone loves data

Data is the lifeblood of any business.  And your data in your CRM is no different.  Good data can empower any business.  While on the flip side, data can also become a blocker for your teams when not managed well.  But the good news is this shows your users have something at stake in making it better.  These are the same people who can become your data champions!

To further the point I am trying to make around identifying your stakeholders.  Your end users play a big part in advising you on what the data actually means (for them).  And it is here you will also need to identify other potential stakeholders.  Does the data have an impact on sales reporting?  How about on your company’s financial reporting?  Or is it used to help serve or fulfil your customers.

I have a question for you ma’am!

As part of any data migration, you will need to get up to speed on understanding the data.  Understand and interrogate the old data.  Analyse it.  Test out any assumptions you might have.  And also think through how you want the data to look once the migration has been completed.

This is a time you are going to need to ask a lot of questions and ensure you (and your stakeholders) fully understand the definitions being used.  If there is something that doesn’t make sense, ask questions!  Questions are your friend!  Really drill down and confirm your understanding.  Leverage the experience of your stakeholders, bring them along the journey too.  And make a note of these questions are you get answers.

One lesson I have learned over many data migrations is no-one has all the answers unfortunately.  Most times you have to really dive into the data and test what you have been told.  This is one area that is a big risk to any project and can end up causing a lot of pain no matter how well planned!

I have been on the tail end of a few really complex data migrations.  Even with the best planning and stakeholders, there has always been an inconsistency somewhere in the data which has you screaming ‘WHY!!!!’   Especially if you are dealing with an older CRM!  (but that is a story for another time!).

The point I am trying to make is it has been down to the project team to really untangle and interrogate the data to make sure it makes sense.

Prepare, verify, test

One you understand the data, you can plan to migrate the data!  There can be a lot to think about at this stage.

In your preparation, you need to map out where you want the data to go.  Are there new fields created to support the migration or are you mapping to existing fields in Salesforce?  Does the data-type match (eg does a number field map to another number field)?  Do you have any validation rules or mandatory fields that might get in the way?  If so what is your plan to mitigate that?  Do you update the data before you move it?

Some of the biggest problems I have encountered in a data migration have all stemmed from a lack of full testing in a sandbox.

If you have access to a sandbox, plan out your testing.   Even if you have Professional Edition, you have access to at least a Developer Sandbox (check here).  Part of your testing needs to include verifying the actual data.  Does to match what you had in the data source?  Do any lookups to other records link up correctly?  Are the number of records the same?  All those questions you wrote down while understanding the data, can be very useful in the testing phase.

I have thrown a lot of questions around there, but hopefully you get an idea of some of the things to think about in this stage.  Testing is another great time to re-engage with your stakeholders.  Heck, even get them to login and run a few test scenarios on the migrated data.

Once you have tested and verified your data, you can start planning the go live and move into production!  Your almost there!

Step back in time

With the actual data migration into a production / live system, you should plan in advance on how you intend to roll back any changes you make.  Even after testing in a sandbox, there could be something which causes you to halt the data migration.  And if that happens, you need to be able to restore the system to how it was before you started.

Take a backup, run reports, hold onto the import files from Dataloader.  Whatever it is, just know how you can go back without causing further issues.

This is doesn’t need to be overly complex, but it will depend on the size of the overall size of the data migration.  Sometime it can just be a simple plan to say ‘if this happens, this is how we move forward’.

Press the button & migrate data!

This is the fun part, and all your planning comes together.  This is where you start the migration and hopefully completed the migration as planned.

Once you have migrated the data into Salesforce, test and verify again.  Repeat the steps you did after you loaded the data into the sandbox.  Test and verify, test and verify!

Are you finished yet?  Good.  But did you disable any workflows/validation rule/mandatory fields?  Did you remember to turn them all back on?

And finally, and most importantly.  Have you communicated the successful data migration out to your stakeholders!? 🙂

Do you have any lessons learned?

As mentioned at the start of the post, I feel there is always soomething new to be learned everytime you migrate data.  One of the projects I am working on at the moment is pushing me to learn something new every day! (More on that in another post though).

Sharing is caring 🙂 Do you have any tips or things you like to plan for?  Share them in the comments section below!

Data security, sharing rules, org-wide defaults, role hierarchies – are all parts of a bigger puzzle, which can feel like a maze sometimes.  It isn’t a sexy topic by any stretch of the imagination, yet defining a robust data security policy is fundamental to all Salesforce orgs.  Let’s dive into what data security is, and how to set it up within Salesforce.

Data security in Salesforce

Data security within Salesforce has a number of facets to it.  You have varying levels of security controls, which can seem quiet complex to start off with.  Though it is this flexibility which actually gives you a lot of control, allowing you to show a specific object/record/field to one group of users while hiding it from another.

The most common form of data security is controlling access to an object or field through the user’s assigned Profile/Permission Set.  For example, you set a user’s access to an object by defining if their profile can either Create, Read, Edit or Delete.  This is often referred to as CRED permissions (or CRUD where the U = update).

Object access is the first level of security a user will need to pass.  If a user doesn’t have access to an object, they will never see any records.  This gives you a sledge hammer approach to control access, but what happens if you need to finesse this?

Say you want to control access to a record based on a specific field value and then share that with certain groups of users?  Controlling a user’s access to an object doesn’t really help you out, does it?

This is where your Salesforce instance’s Org-wide Defaults (OWD) & the sharing rules come into play.

Org-wide Defaults (OWD)

Org-wide Defaults or OWDs for short, is the next level of security and will allow you to define what the default level of record security is per object.   If you were to try to visualise it, most people draw it out as a funnel.  This is because you first start by restricting the Org-wide Defaults of an object, and then extend access from that default.

What do I mean by first restricting access?  In Salesforce you have three levels of default record visibility per object:

• Public Read/Write – all records are publicly available to read & edit for all users who can access the object.
• Public Read Only – all records are visible for all users who can access the object, but not edit.
• Private – no records are visible/editable for users who can access the object.

Now the above rules have two key exceptions. The first is when the object in question is part of a Master-Detail Relationship.  If it is and it is the child object, it will inherit the OWD from the parent object(s).

The second ‘gotcha’ is when the user is an owner of the record in question.  Ownership of the record will grant access to the specific user who owns the record, providing their profile grants them access to the object.

Role Hierarchy

Role Hierarchies within Salesforce play an important part in record visibility.  As they allow you to build in general record visibility for those higher up the hierarchies branch.  The most common example would be when you want a manager to see what their team is doing within the system.

This is important to note as when you set your OWD to either Public Read Only/Private, the role hierarchy needs to be taken into account to ensure that relevant hierarchy above the owner of the record still have the relevant access.  When setting the OWD, you do this by ensuring ‘Grant Access Using Hierarchies’ is enabled.

(For more information on Role Hierarchies, Salesforce Help & Training article)

Sharing Rules

Now we have built the foundations for who can see what, we need to extend access as required to the various types of users.  This is where Sharing Rules comes into the picture, they help you extend the default settings from above.  One example could be where a team of sales users needs access to each other’s records.  If the OWD is Private, you would need to extend the sharing of records within that team.

Sharing Rules within Salesforce give you two types of control, based on either the:

• Record owner (eg user ‘Sharon Williams’ owns the record), or
• Criteria based sharing (eg if field ‘Record Type’ equals ‘North America’)

This allows you to define how a record gets shared.  And this can be to a group, specific role or a role and all of its subordinates in the hierarchy.  You then pick how to the level of access to finish up (read only or read/write).

There are some nuance’s with sharing rules, and limits are enforced on the number of rules you can have by Salesforce.  For example you can have up to 300 sharing rules on an object, with 50 criteria based sharing rules.  And only specific field types can be used for Criteria based sharing rules.  To view the considerations, go to the Help & Training article for more.

Manual sharing

Manual sharing is an additional control which allows you or the user to manually share a record.  This is done by clicking the ‘Sharing’ button on a record (if it is on the page layout).  This is where you can also see the sharing detail, which will display who has access to the record, what that access is and why they have that access.

Final words

Hopefully that helps you control your org’s data security, as you can see there is a lot of detail and thought that goes into ensuring the right people see the right data.  As an additional topic, there are also Apex Sharing rules which can give you even more control over who can see records, but I won’t cover these in this post.

There are also some great resources are available from Salesforce if you want to dive into this topic further.  I found this video walk through provided which is ~5min video.  It shows you how to setup the rules we discussed here in Salesforce Classic. There is also an awesome Trailhead on the topic of Data Security, which you can find here.