clean data

  • A Tale of Salesforce API Limits

    Everyone loves data, I am sure of it…  The key difference though, is most people in their right mind, simply wouldn’t admit it.  Data is useful, particularly in this day and age, in what is now commonly referred to as the Information Age.

    Now, you might be thinking I am crazy – and that is probably true – but data is the life-blood of modern-day organisations.  So what happens when can’t query, transform or extract your data because you hit Salesforce API Limits?

    gasp in horror, hitting API limits during a data migration

    Intro to APIs….

    Ok, ok – I hear you.  What the hell is an API limit?  For that matter, what the f*** is an API?  And why should you care?

    Starting at the beginning (feel free to skip this section if you already know this!)…

    An API, or Application Programming Interface, is at its most basic a way programs or systems talk to each other.  I am way over-simplifying things here.  But an API ultimately allows one piece of software to communicate with another piece of software in an agreed way.

    data apis, allow computers to talk to each other
    APIs allow computers to talk to each other

    Software, like Salesforce, is built to be a relatively open platform.  And they do this by using a number of APIs to allow other bits of software to communicate and link to it.  These links allow developers to do things like query, extract, create or update records within Salesforce with relative ease.

    Salesforce supports SOAP API, REST API, BULK API, Streaming API, Metadata API, etc….  So there is a number of ways to access what you need.

    Tools you may use as an Admin everyday, use a variety of these APIs.  For example, DataLoader uses the SOAP/REST/BULK APIs depending on its setup.  Likewise Workbench supports APIs too, as it can connect and manipulate data, query the fields, test API connections within Salesforce.

    Even tools like Salesforce for Outlook use APIs to query records and insert attachments.  And as an admin this is why you should care.  These tools, along with many others, use APIs to connect to Salesforce.

    But with great power, comes great responsibility…

    So with all these connections into and out of Salesforce globally, surely that can slow down the platform?

    That is what brings us to the Salesforce API limits.

    Salesforce is a multi-tenanted platform, meaning no-one company has an exclusive server/computer/machine setup for them, everyone shares the infrastructure which drives the platform.  So being cloud-based, Salesforce needs to ensure the performance of the platform works for everyone.

    To do that, Salesforce impose limits.  Governor Limits, Storage Limits, Per User Limits….  All based on the version you have bought from them.

    These limits are designed primarily to cap usage of the platform, ensuring stability of the platform and still give a fair amount of flexibility.  All the while, also providing a potential upsell opportunity for extra Salesforce services (eg Data Storage limit).

    Salesforce API Limits

    Among these limits though are API Limits.  And even though you normally shouldn’t notice there are limits in place, when you do hit the limits – especially when you hit them unexpectedly – it can be a little jarring…

    System.Web.Services.Protocols.SoapException: REQUEST_LIMIT_EXCEEDED: TotalRequests Limit exceeded.
       at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

    This is the error which stops all work via APIs… If you rely on APIs to connect to your systems – you really don’t want to see this message!

    In this example, a connection using the SOAP API to connect into Salesforce has reported the Request Limit has been exceeded…

    Did I do that? API Limit in Salesforce hit...

    Diagnosing API usage…

    As an Admin, to diagnose the issue the first port of call is to analyse and find out how you hit the limit.

    Every org has quick look view at the API limits.  So go ahead – check for yourself.  Go to Setup | Quick Find | System Overview to see what you have within your org (if you have access to the Setup/Config of your org that is…).

    Salesforce API Usage within System Overview
    Not quite there yet… But closing in on the daily API limit!

    Firstly, all this really confirms is what has happened.  Like a car not starting, you have confirmed there is a problem – but you haven’t diagnosed the issue.

    When is 24hrs, not 24hrs?

    And secondly, when is the 24hrs?  Is it a rolling 24hr period, is it the start of each day local time or even the start of the day in San Francisco?

    Well, the good news is this limit is a rolling 24hr period.  Meaning it shows you the last 24hrs, from the time you are checking the limit.  This is important to know, if you have a lot of batch processes which might be scheduled at specific times of the day.

    Is that all the information you need?

    Does that really help you though?  Overall it doesn’t really give you much insight as to what has really happened within your org.

    So, what about Company Information?  There is an API Request Limit there too…. (Go to Setup | Quick Find | Company Information)

    API Limits at 99%
    Company Information in Setup

    Nope… still not much help!  In this example (yes, two different orgs), we can see the API Limits is 98.9% at the limit for the day.  So you will likely start to see problems with API connections…

    API Usage Last 7 Days

    On the Help page, Salesforce referenced an ‘API Usage Last 7 Days’ report, but didn’t mention where I could find the actual report!

    The help article advises that you need: ‘View Setup and Configuration’ permission on your to see this report.  I definitely have this permission within the org…. But I still couldn’t find the report: 

    Then I was reminded about the fact Salesforce comes with some generic but useful standard reports.  And guess what is included in the folder!?  Yep, an API Usage Report 🙂

    But you need to go via the Administrative Reports folder in Classic to view it…  Go to Reports | Folders | Administrative Reports:

    API Usage Report in Salesforce
    Salesforce API Usage in Last 7 Days Report

    If that doesn’t work for you, you can also use this URL hack (thanks to Joshua) to jump to the report within your org:

    https://na1.salesforce.com/00O?rt=104&retURL=%2F00O&c=UN&c=FULL_NAME&c=EM&c=CID&c=TS&c=CC&duel0=FULL_NAME%2CUN%2CEM&scope=organization&details=yes
    Replace the na1 with your instance.  Below is the same url without the na1.salesforce.com
    	
    	/00O?rt=104&retURL=%2F00O&c=UN&c=FULL_NAME&c=EM&c=CID&c=TS&c=CC&duel0=FULL_NAME%2CUN%2CEM&scope=organization&details=yes

    Diagnosis, complete…

    Well… Almost….

    The report gives you a great breakdown to see what/who is using up your Salesforce API Limits over the last 7 days.  But it doesn’t fix the overall issue.  The fix itself can be as complex as the software/tool that is causing the API Limit to be used up!

    But at least you can now diagnose the issue and decide if you can fix the issue, alternatively if it is something you aren’t likely to be able to fix/reduce you can sometimes buy additional API Limits from Salesforce by speaking with your AE…

  • Cutting the Clutter: Maintaining a Clean Salesforce Org

    Maintaining a clean Salesforce org, doesn’t need to be a battle. Recently, I was watching a great webinar by Kelly & Leanne entitled ‘Cut that Clutter‘. And it got me thinking about how the problems faced by a cluttered Salesforce, and how it can easily consume an Admin’s time and effort. And it not only affects us as Admins, but also our end users! So in this post I am going to recap some of the awesome tips shared in this webinar, and also see offer some additional FREE tools to tackle the problem that so many of us face!

    The War Against Clutter

    Ok, ok. I admit that this header is full of hyperbole.

    Maintaining a Clean Salsforce, shouldn't be a battle...
    Don’t raise the white flag, in the war against clutter!

    But a cluttered Salesforce creates a lot of frustration and anxiety for me. And I assume most it does for most of you reading this too! My personal vendetta against clutter drives me to ensure I am always improving the org for my end-users…

    I have previously written about how we started to tame the Technical Debt beast haunting our primary orgs and removed over 2 million records from an org (hint: very manually).

    But there is still so much to do…

    It is a seemingly never-ending fight. But as Admin’s we are always looking for tools and resources to help us in our day to day Admin Superhero duties. And to help us in maintaining a clean Salesforce org…

    Cut that Clutter! – The Recap

    Now before we get much further. If you have a spare 30mins I strongly recommend that you watch the webinar as I am only going to briefly summarise it here…

    The session covers the Three-S’s. These are the primary areas to focus on, to ensure your CRM is kept in tip-top shape.

    • Security – making sure you know who can see what in your CRM
    • Structure – does the setup of Salesforce ensure data security and meet any data governance requirements
    • Strategy – how to plan and scale while ensuring you don’t have to keep doing ‘big clean-ups’ each year

    If you want to hear more, then please check out the video.

    Cut that Clutter: Resources mentioned

    Next up, the ladies mentioned some great tools to help you in maintaining your Salesforce org.

    From Salesforce:

    • Salesforce Optimizer (aka Optimiser in non-US/Canada countries 🙂 ) – I am in LOVE with Optimizer reports. It is such an amazing tool to help you analyse and understand where the Technical Debt is likely to be hiding within your org. This should be your first port of call, in maintaining a clean Salesforce. That is how much I love it!
    • Security Health Check – helps you understand any vulnerabilities you may have within your Salesforce. This covers areas like Password Policies, Critical Updates, etc.

    From AppExchange:

    • Field Trip – this tool is one I install in every org I have managed now for a number of years!! It is a great tool to help analyse and understand just which fields are being populated and used by your end users. It is worth noting, that if you have a field that is always updated automatically by a trigger/workflow… Then it will obviously show as being used, even if that trigger/workflow update isn’t actually required. But overall it will help you understand your org in very tangible way.
    • The Permissioner – can help you when mass assigning/removing Permission Sets from your users.

    From Trailhead:

    Additionally the ladies have set up an Admin Trailmix.

    This covers a number of modules covering: Salesforce Profiles/Permission Sets, User Authentication, Data Quality, Data Management and finally Reporting & Dashboards.

    Help with maintaining a clean Salesforce org

    Extra, extra! Two more tools to add to your Salesforce Cleaning toolkit…

    Now for the bonus round.

    There are always so many tools and ideas out there helping admins when maintaining a clean Salesforce org. And I am only skimming the surface with these next two tools…

    Compare Permission Sets & Profiles

    When watching the webinar, albeit not live, I started shouting at my screen.

    During the Security section, there was a point around Profiles/Permission Sets. As an admin it is a mammoth task to compare all profiles/permission sets and what they might grant access to within your org. This can be kryptonite to Salesforce Superadmins…

    There was a recommendation to switch off Enhanced Profile View, and then compare the permissions… But why do that? Especially, when there is a secret weapon at your disposal?

    Perm Comparator by John Brock is that secret weapon!

    Seriously… More people need to know about this tool! And I am not even on commission! 🙂

    Stop duplicates in their tracks…

    Salesforce hasn’t always been an admin’s best friend when it came to cleaning an org…And without getting all ‘back in my day’-ish…

    But there was a time Optimizer, Security Health Check and those tools simply didn’t exist.

    There was also a time Salesforce didn’t have an easy way to prevent duplicates… Admins had to either buy other tools to identify and manage duplicates, or create complex formulas and validation rules to try and prevent exact match duplicates.

    But when planning your strategy for maintaining a clean Salesforce, you should investigate the in-built duplicate management tools from Salesforce.

    After all, what good is cleaning up your security (profiles, access policies, passwords) and clearing out fields you don’t use any more – if your end users are still swimming in duplicate records?!

    The in-built feature will take care of the basics, but depending on your use case, there may still be a reason to buy a tool like Cloudingo or DemandTools (just to name a few).

    What is in your toolkit?

    De-cluttering your Salesforce can be so rewarding!
    De-cluttering can be so rewarding!

    As I mentioned I only skimmed the surface here… And this is a topic I can (and will likely) write more about in the future. I have rambled more than enough for now…

    So to wrap up the post, feel free to add any other suggestions or recommendations for your ‘Cleaning Salesforce Toolkit’ into the comments section below.

Back to top button