Things I learned developing Search Software

Most software starts with someone who is frustrated and is looking for a solution that does not currently exist or if it does, does not quickly solve the problem. The following insights are from 18 years of developing over 100 tools to make it more efficient to complete my projects.

Is there a market for it? 

I must say that 90% of the time, there is no commercial market for most tools. During a presentation at SMX Advanced last year, I showed a list of some of the nearly 70 custom tools I have developed to do my job. I use at least a dozen of them on a weekly basis, and of half of these, there is not a commercial equivalent or one that has the specific functions I need. You might ask why if you need it why don’t the multitudes of SEOs need it? That is always my question, why don’t they need it? It is easy to suggest that we have different approaches. I tend to be data-driven or want to find patterns to speed up the process. The real answer might be below in the “easy button” section.

Here is a prime example, a big challenge for website owners is minimizing the lost traffic from site refreshes and migrations. During these events sites may lose as much as 80% of their traffic post-launch. With such a negative impact on their business, you would assume that there would be a market for tools that could mitigate this loss.  

We at Back Azimuth were doing a lot of large migrations and acquisition integrations so we built nearly a dozen tools to facilitate our workflow and demonstrate the potential for loss if quality control was not present.

These applications were not pretty but highly functional and after presenting the methodology at a couple of development and SEO conferences all the signals showed there was a significant need for the suite of tools. Who would not want a solution that, if used properly, makes migrations and site refreshes nearly disaster-proof? With all of the interest and the size of the business problem, we merged them together, improved the user interface and the Site Migration Suite was launched.

Unfortunately for us, there was little there was/is little to no interest from either the search or the development community. We had a few SEOs who wanted to demo it and, ultimately, about a dozen subscribers that raved about how it saved them from disaster but that it created more work than they cared to do. It was not a lost opportunity since we have made significantly more money fixing migrations at nearly ten times the rate of the tool license.

People want an easy button 

I guess it is not unexpected that people want tools to do everyone automatically and present pretty reports. Many don’t want to think or interact with them just want them to do everything and just puke out a successful result.

Of the last 100 emails related to HREFLang Builder the words that appear most are “automated,” “no effort,” and “set it and leave it,” related to the creatures people want to know about the tool.  We get asked almost daily why we have not added AI into the setup to detect the various formats that a site uses for its URLs.  We do have it, but most users have complex setups that require human intervention.   

People refuse to read user guides. 

Unless it is painfully easy to setup and use, which is nearly impossible for the first few versions of a tool you need to assume you will spend a lot of time on setup.

Onboarding has to be painfully simple and as few steps as possible, accounting for any many variances as possible since people will not spend the time to learn how to do it.   We are rebuilding the entire setup function of HREFLang Builder since nearly 100% of those that subscribe do not read the “read me first” documents and have problems getting set up.   This requires us to spend a lot of time hand-holding people to get even the simplest of sites set up.  

It seems to be such a problem that my friend Dan Weingrod has built a successful consulting practice using Design Sprints to help identify problems with onboarding and getting people to successfully set up tools.  

How many customizations do you do?

Nearly every search industry tool developer has needed to build dozens of functions to account for the craziness and dysfunction of what people pass off as websites. For HREFLang Builder, we have over 100 functions integrated to work around issues that violate some of the most fundamental web dev rules. In one regard, this has made it the most powerful tool in the market, but often we cannot innovate for advanced functions due to dealing with customization to account for a large customer.

We look at customization through benefit tests. Simply, if this modification this something that we can use for many customers, and will we be a better tool because of it we will spend the time and money to do it. For example, due to the inability of a client to build a complete set of URLs, we needed a method to import URLs from 5 different sources. Initially, we assumed the best source was CMS-generated XML sitemaps but many systems have problems or different methods between CMS. In the end, it worked it, it helps clients with complex structures to use the system easier.

However, if the customization is for a single customer and there is no benefit to others, then the customer must change their challenge or pay for the customization. We have turned away a few significant prospects that assumed we wanted their business and would do anything to get it. It is often better to say no than take on the risk of developing custom features and not having a real commitment from the client.

People don’t use the features they request

I assumed this one, but until we added tracking, we could not prove it.   We have done a lot of customization to a few tools based on customer feedback.   We have at least one report in each of the main tools that have been requested by multiple clients that have never been used by them.  In one case it was deemed mission critical to have it, but they never clicked the link to generate it even though they wrote the specification and approved the final live version.   We have been less willing to accommodate feature requests before onboarding a new client. We have one prospect that dangled 200 sites and wanted a custom API connection. We decided to build it so that we can integrate it with other tools. In the end, they went with an agency to offshore the function manually.

People don’t want to login 

Along with the “easy button,” people really don’t want to use tools or log into them.  Several years ago, when I showed him DataPrizm, Conversion Expert Bryan Eisenberg told me that do anything I can to eliminate or reduce the need to login to the application.  With DataPrizm using the tool is what it is made for, and just sending reports diminishes the value.  

He was right, especially today. Nearly every user we have now just wants us to push reports out to Domo, Google Data Studio, or some internal system. We even had one client pay us to develop an API so they can customize their exports and integrate them into their dashboards and never have to use the tool itself.  

People care too much about pretty over-function. 


I am not talking about something is complicated to use or lacks basic usability the feedback is not really what you might expect. Feedback from users especially on workflow is critical but the colors, button placement even fonts people want changed. Most of the time the things they complain about have nothing to do with the actual function of the tool.

We often roll our eyes at people on the house hunting shows that walk in and decide not to buy the house due to the color of a room or the carpeting—nothing to do with any function, price, etc but the easiest and least expensive things to change.  

We have had people tell us they would not sign up unless we changed the color or removed a logo.   We had someone that said they had 3 basics account clients, but before he would sign them up, he required a custom URL and his logo in the tool since he was showing it to clients. Sure, for $75 a month, let me get right on that.  

Who will you upset?  

This has been an interesting learning curve with all the tools. In many cases, you would assume people would embrace tools that made their work or lives easier.  It comes down to who and what process you will disrupt.  The more you change a process and highlight inefficiency the more people will reject the technology.   We have had some negative comments from various consultants and agencies that did not want to use automation, especially for the hreflang functionality that they were doing manually.  

One of our biggest detractors for HREFLang Builder was an agency team that was spending between 20 and 100 hours a month doing them manually for clients.   They did not want to lose those billing hours.  In most of the cases, the client wanted them to do something more constructive with the time and just license the software. 

Getting Paid 

In the agency example above, you might assume that a tool that costs less than a quarter of the labor hours to do the project would be an easy sell to agencies, but that is rarely the case. From my time at Ogilvy and WPP getting agencies to pay for tools has always been a problem they just cannot convert billable hours into tool expenses. Most agencies live in a world where they must bill everything to a client including the cost of or at least a fraction of the tools they use on the client’s behalf. If it was not budgeted for in the original scope or cannot be directly billed is nearly impossible to get adopted.  

This is often the opposite of how most clients and other rational people think. If you go to get your car serviced by a mechanic, you don’t pay for them to use their tools to fix your car. You expect them to have the right tools and diagnostic equipment to do the job as part of the price you pay for the repair.  This creates a challenge for everyone.  

You need to have flexible payment options either by billing the client or giving the agency access to the solution creates a challenge for most subscription and login applications.     

So what to do with your excellent tool idea?

The best approach is to create that MVP and go out and test it to see if people will adopt it.  If you don’t get people lining up and raving about it, most likely the market won’t adopt it. If it saves you money and gives you a competitive advantage, then build it out for your team to use it. If you can crack some of these key challenges when you launch your application, you will be ahead of the curve and have a better chance of success.