Startup Lessons: The Handoff from the Remote Dev Team

In my CTO as a Service consultancy, I tend to use various remote development companies to write the code for my clients. Even though I still code after all of these years, I am usually not cost-effective for my founders in a coding role. (But I still do the architecture, all of the set up on the cloud platform, and occasionally write some critical code.)

When the engagement with the development team ends, there is a wrap-up process that takes a few days to accomplish. If you have a remote development team, then you should allow them at least 3 or 4 days to perform these final items.

I would love to hear about your own list that you go through in order to wrap up an engagement.

(Note: Even though I mention AWS in this article, you can substitute any cloud provider, such as Azure or GCP.)

Tasks for Wrapping Up


  • Read, edit, and check the documentation that the consultants write so that it can be handed off to another development team
  • We need the following documentation
  • Architecture diagrams, including integrations with 3d-party services
  • Structure and flow of the backend and frontend code, including interaction with 3rd party services
  • Deployment instructions for all environments
  • Troubleshooting guide


  • Code review (if time allows)
  • Go over the Jira to see what was not done and to see if anything was left in a partial state
  • Make sure that all tests are passing
  • Making sure that all code has been merged to Main
  • Make sure that we have access to all secret keys and PEM files
  • Make sure that we have access to all licensing and credentials for 3rd party libraries and systems
  • Doing a deployment on both test and production to make sure that we know how to do it ourselves


  • Checking out the production database to make sure that everything is smooth
  • Make sure that all routes are correct in Route53
  • Tighten permissions on S3 buckets
  • Closing down all open permissions on the various machines
  • Possibly removing the consulting team from Github, AWS, 1Password, Twilio, Stripe, etc


  • Install Monit on all machines
  • Setting up various CloudWatch alarms on AWS

CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. I work with all sizes of companies, but I love helping startups realize their visions. Services include hiring and leading development teams, architecture, code and architecture reviews, roadmaps, coding, speaking with investors, etc.

CTO as a Service: A Recap of the First Year

Looking back on my first full-year of CTO as a Service…. some statistics.

I had 13 clients. 6 are currently active.

Out of the other 7 clients, I helped 3 of them hire their permanent CTO or VPE, and I transitioned out of those companies. 2 were short-term advisory or architecture review gigs. 2 companies called it quits because the founders ran out of funding.

The clients were from EdTech, HealthTech, SelfHelp services, RealEstate, LegalTech, large investment banking, FoodTech, social media, investments and trading platforms, and financial help services.

Client 14, an InsurTech startup, just signed with me this week. I am most excited to get exposure to AWS IoT Core.

My goal was to keep myself busy 20 hours per week in my “semi-retirement”. I easily met this goal.

All of my clients were on AWS. Not a single client was on Azure or GCP.

Languages and platforms used were mainly C#/ASP.NET Core and Typescript/NodeJS.

Databases were split between MySQL, SQL Server, Postgres, and Mongo.

JIRA, Confluence, and Slack were other tools that were heavily used.

I am really looking forward to 2020. All in all, I am happy with my decision to leave the BigCo environment, and devote my time to helping small companies get to the next stage.

#cto #startupcommunity #ctosummit #startups

Creating an MVP with Limited (or no) Budget

Marc Adler

CTO as a Service

On almost a daily basis, I receive email from non-technical startup founders who are looking for a CTO. A good percentage of these founders have little or no money to devote to their startup, but they have a dream, which is always the first step.

The solicitation from these founders usually starts off with “I am looking for a technical co-founder, but I can only pay with equity for now.” I wish that I could devote a slice of my time to assist each and every one of these founders, but in reality, I do not work for equity anymore. I would gladly give my technical advice for no charge for certain social causes that I believe in, but my current business model is to charge an hourly rate or a monthly retainer.

Do they need a CTO right now? Do they need a technical co-founder at this point? This is one of the most popular questions that I get at my monthly Startup Lean Coffee meetup.

First, the startup founder needs to consider what they are going to build as a first step. Are they looking to create a Minimum Viable Product (MVP)? Or are they looking to develop the complete working product? The MVP can have just enough functionality in it to prove out the idea and to attract further investment. Depending on the answer to this question, they may want to bring in an experienced technologist for just a few hours to sketch out what the technical architecture will look like.

Most of the time, they just need someone to code the MVP. If they have enough budget, then they could use a CTO-type advisor like myself to guide them through the technical aspects of creating their application and pitching the MVP to investors. But, often time, they need some coding help that can be paid for in equity rather than in cash.

If you are a startup founder with a limited budget, and you need to develop your MVP, don’t despair. There are a number of routes that you can take to develop a basic MVP that is designed to attract more investment.

Strategies for Creating the MVP


In order to test out the viability of your idea, you can create a “demonstrator” of your idea. This demonstrator can be as simple as a few web pages stitched together, using fake test data that is stored in a few files on your computer.

No-code or low-code development environments have been around for many years. These platforms let you drag-and-drop visual elements onto a canvas, and create a fully-functioning webpage. You can create a number of pages and hook them together through various kinds of user interactions, such as clicking on a link or a button.

The nice thing about these platforms is that you do not have to code anything yourself (or, perhaps, very little coding may be needed if you need more complex functionality). Some of the platforms will let you automatically integrate with various third-party services. You may want to hire a graphics designer to create some custom artwork that you can incorporate into your demonstrator. You may want to consider hiring a User Experience (UX) specialist to make sure that your application flows together in a logical and pleasing way. But you will not have to spend any money on hiring a developer.

Try to code up a demonstrator version of your product using a web-based design platform such as Balsamiq, Sketch, Figma, or InVision. A more advanced low-code platform whose popularity is increasing is Bubble. There are integration platforms such as Zapier that can be used to hook up various third-party services.

Teach Yourself How to Code

There is nothing like getting into the weeds and coding up what you want. Many people have taken a programming course or two in school and might be a bit rusty. If you have not learned how to code, there are courses that you can take at local community colleges, night schools, and online. For example, Udemy has a course in Introduction to Programming. Other MOOCs like Udacity and edX have additional courses. If you want to self-pace yourself and get a wide variety of content, I like Pluralsight.

The courses mentioned above are fairly inexpensive. Once you learn to code, you don’t have to pay another programmer, unless you need some sort of special skill.

Approach a Recent Bootcamp Graduate

There are many coding bootcamps that new programmers go to in order to get intensive training in coding skills. These bootcamps, like Flatiron School or General Assembly, often last for several months and teach the new coder a wide variety of technical skills. The bootcamps will try to find their new graduates a programming job, but these graduates often face the chicken-and-egg problem – many companies want to hire developers that have some amount of experience, but how does one get that experience?

These graduates are hungry for some experience that they can put on their resume. They might be receptive to an equity-only arrangement for a few months while coding up your MVP. There is nothing like having the satisfaction of bringing up a website and saying “I wrote this”.

My only word of caution is that these graduates learned a lot of new technologies in a short period of time, without much practical experience in applying what they learned. Be wary of “resume-oriented development” where the new graduate will want to use a certain technology because they want it on their resume. An architecture and/or code review by an experienced CTO will help identify inappropriate technologies that might come back to bite you later.

Use an Offshore Contractor

You can usually find an independent coder in countries like Russia, India, and Vietnam who charge a fraction of what you would pay a developer in the USA or England. You can find good developers for as little as $20 per hour, or you can negotiate a fee that is outcome-based. 

How do you find these developers? There is the ever-popular CraigsList, which I personally would never use, as you are subject to being contacted by lots of scammers. You can contact a local college and post something on their job board. Recently, a number of platforms have emerged where freelance developers advertise their availability while you advertise your project.

A popular platform for finding these types of coders is Upwork. You post your project on Upwork, and you get “proposals” from developers who are interested in working on your project. After examining all of the proposals and interviewing the various developers, you pick the developer who you would like to work with. Developers who have done projects before through Upwork are rated by their customers (and the developers also rate their clients).

One word of advice – Upwork takes a certain percentage of your payment to the developer. This percentage decreases as the developer bills more hours to your project. Keep in mind that the developer will not receive 100% of your fee, so you may want to adjust your fee so that the coder gets a fair wage.

Incubators and Accelerators

There are various incubators, accelerators, and Angel Investment funds that will fund you if they like your pitch. In New York City alone, there are probably a few hundred of these types of places that you can go for some initial funding. There are giant incubators and accelerators, like Techstars and Y Combinator. Many startups apply to these incubators, and the acceptance rate is very low. There are more boutique accelerators like Brooklyn Bridge Ventures. If you are a woman-owned startup, and especially a minority woman-owned startup, then Pipeline Angels is a great place to consider. And some universities, like NYU, have their own accelerators that are designed to fund their graduates.

I help run the Startup Lean Coffee monthly meetup at Betaworks Studios in New York. Betaworks is a space where entrepreneurs gather and exchange ideas about their startups. They also host almost nightly meetings for the startup community.

Some of these accelerators will want to see a senior-level tech person that you are associated with before they will give you funding. Techstars is well-known for this requirement. If you need to have that kind of CTO-like person involved with you when meeting investors, then this is a service that CTO as a Service can provide.


There are a number of grants available at the state or local level for certain startups. These grants usually come with no strings attached. You do not have to give away any equity in exchange for these grants. Of course, there is a lot of competition for these kinds of grants, and they are usually awarded to startups that are working on ideas that will improve society in some way.

Final Words

If you are a startup founder who has a great idea, then congratulations …. you have taken your first big step. Even if you do not have any money right now to take your idea to the next step, there are several different avenues for you to pursue in order to develop your MVP for little or no money. Some of these avenues will require you to spend a little bit of money, and you may be able to get that money through your savings or through Friends-and-Family investments. Some of these avenues will require you to give up some equity.

No matter which route you decide to take, make sure that you always have a senior technical person who is watching out for your interests by your side during the early stages.

CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. Please consider CTO as a Service to be your senior technical advisor on any projects that you might develop.

Marc Adler

CTO as a Service

November 2019

Thanks to Kathy Keating, George Sudarkoff, Dave O’Flynn, Glenn Proctor, and Adrian Howard for their suggestions.

Startup Founders – Questions to Ask Offshore Dev Shops

Marc Adler

CTO as a Service

(Note: This article is written for the non-technical founder of a potential startup. Nevertheless, the questions listed below can be useful for even the most experienced IT executive who is in search of a new remote development shop. In addition, the article assumes that the founder is based in the USA.)

So, you have decided to break free of corporate life and pursue that dream of turning your big idea into a startup company.

Congratulations !!!!!

In all likelihood, you are not a technical founder. You may have taken a few programming classes in college, or maybe you even spent a few thousand dollars and went to one of those Coding Bootcamps that teach you how to code in 8 weeks. But, in reality, you probably don’t trust yourself to sit there and write an app that will be the lifeblood of your startup.

You have probably squirreled away a bit of money from each paycheck in order to live your dream one day. You may have even gotten some friends and family to chip in a few dollars. You might quit your day job to devote full time to your new startup, or perhaps you will still work that job while waiting for the app to be developed.

So, what are your next steps?

Hopefully, you will have “rented” a CTO for a while … someone like me … to guide you through your next steps. After all, you will need someone very technical who represents your interests so that you will not end up with nasty smells in your shiny new app. 

(Read this article about code smells and remote development teams.)

Building Your MVP

Even if you don’t have an interim CTO, you still need someone to develop your app. Most startup founders will develop an MVP, which is short for Minimum Viable Product. This is a lightweight version of your full app which is designed to provide the necessary functionality while whetting the appetite of potential investors.

You hopefully have at least $50,000 in funds which will be used to develop the MVP. That amount is the finger-in-the-air estimate that I always give to my clients. You probably want a web-based version of your app, as well as a mobile version (iOS and, possibly, Android).

The more that you can do yourself, the more you will be able to devote to the development costs. For example, you might try to write up the business and functional requirements yourself. Instead of sketching out your wireframes using pencil and paper, you might want to use an online design tool like Sketch, Figma or Balsamiq to draw out the entire user experience.

You have your idea for an app, you have some idea of how the app might look, you have your business and functional requirements written down, you might have a technical architecture, and you have some money. The next step is to get some people to actually code up the app and set up the infrastructure that the app will run on.

What Kind Of Developers Do You Need?

The most important developer that you need is the backend or server-side developer. This is the developer who will be interacting with the cloud infrastructure (assuming that you are not going to buy your own servers and that you will use a cloud provider like Amazon Web Services, Google Cloud Platform, or Microsoft Azure). This developer will develop the data models, the corresponding database schema, and will interact with a database platform like Postgres, MySql, SQL Server, or MongoDB. The backend developer will write all of the business logic of the application, will develop REST-based APIs (so that other developers can interact with your application), will handle authorization, authentication, logging, and monitoring.

As you can tell, the backend developer is the real workhorse. Their knowledge has to span multiple technologies and different layers. They will hopefully have direction from your interim CTO, who is hopefully a very technical CTO that has architected your system. The backend developer may present some key person risk, because if that developer leaves your project, then it may take several weeks to train a new backend developer on the codebase.

If you have enough funding, it is desirable to have two backend developers who can divide the sever-side tasks equally among themselves. This also will alleviate some of the key person risks, but only if the two developers do not work in complete isolation. 

The other half of the equation is filled by the frontend or User Interface developer. This is the developer who writes the visual interface of your app. They have to know technologies such as HTML, stylesheets, and they may develop with technologies like AngularJS and React. They have to write REST API calls to fetch information from the app’s backend and present the data to the user in a pleasing way. The frontend developer might be called on to develop the User Experience (often called the UX), which is the overall flow of how a user interacts with your application.

Ideally, the frontend developer can write code for both a browser-based user interface and a mobile interface. You want to find a frontend developer who has the cross-platform experience, and who knows the ins and outs of iOS and Android. They might know technologies like Xamarin and React Native.

There are other players in the development process that you might end up needing.

First, there is the QA Lead. You want someone really going through your entire app, workflow by workflow, before you release the code to the user community. Developers can try to write automated tests, but these tests will only provide a little bit of coverage. You, as the founder, can do the testing yourself, but an experienced QA Lead will know all of the hotspots to explore in an app.

Second, you may need a graphics designer. You cannot expect your frontend developer to be a graphic artist as well. Who is going to design all of those icons, who is going to come up with a good color scheme, who is going to draw all of those custom images? A good development shop might provide you with a graphics designer that will be on call.

Third, you may need a local project manager. You hope that the two developers (and maybe the QA Lead) can manage themselves. You might even want to manage the project yourself at the beginning. And your rented CTO might do the same, although he might be overpaid for that job. As the founder of a startup, your main job is to build your company and raise additional funds, so project management might not be worth your time.

Fourth, if your application is going to be doing any kind of recommendations, or you are thinking about Machine Learning in order to improve those recommendations, you may want to hire a data scientist.

Lastly, a development company might try to upsell you the services of their own CTO, CIO, or Chief Architect to do code reviews of their own developers. This might be good, but you should always have someone on your side to do this, someone who is vested in the success of your startup.

The Money Part

Let’s assume that you have $50,000 that you are devoting to the remote development shop to develop the MVP of your app. How will you spend that money?

Let’s do a bit of math.

My finger-in-the-air estimate is that a proper MVP will take 3 to 4 months to develop. This means that you have around $12,000 to $15,000 to spend each month, which translates to about $3000 per week. If you divide that $3000 into two developers, this means that you are spending around $1500 per week for each developer, or $300 per day. This is about $35 to $45 per hour for a developer.

Where are you going to get a really good developer for $35 an hour?

Most USA-based development shops will charge at least $75 per hour. Many of the really good shops that I have worked with will not take a client unless that client spends at least $30,000 a month with that shop. With your current budget, this means that you will only be able to use that development shop for about a month and a half.

You can go to an online job platform such as Upwork, which is a platform for individual developers to connect with companies that need development services. These developers usually work remotely, and you would have to coordinate all of the various individual developers that you obtain from Upwork. You would also need to be able to properly vet these developers yourself.

You can try to hire someone locally, but in all likelihood, you might be getting a student or someone with a not-so-great track record. Even recent college grads can make way more than $35 per hour. To be honest, I might be a bit suspicious if someone local was to charge $35 per hour.

Your final choice is to use a remote development shop, one that is probably in a country like Russia, Ukraine, Belarus, Poland, Romania, Colombia, Costa Rica, and Vietnam. These development shops will be fully-stocked with the talent that you need, at the price that you need. You can easily find a good senior-level developer for about $5000 per month.

Now that you have decided to explore the possibility of hiring a remote development shop, how will you choose among the thousands of them out there, all with seemingly the same kind of talent? How will you, as a non-technical founder, be able to properly choose among all of the companies who are answering your call for help?

Hopefully, your rented CTO will help a lot and will ask the hard questions to each of the shops. But, if you want to do it yourself, you have to be prepared with a list of questions that you want to ask each of the shops.

Interview Questions for the Remote Dev Shop

And now we come to the gist of this article. Below are the kinds of questions that I will ask the various remote development companies in order to help my client build an app.


  • What are your costs for staff?
    • by skillset (developer, QA, UX/UI, Project Manager)
    • by skill level (senior, junior, mid-level)
  • Will you work on an outcome basis, or are you strictly hourly?

Of course, the price is the main make-or-break decision here. Startups have a limited amount of funds. 

We cannot hire a development team that is outside of our budget. No wiggle room here. We need the best talent for the least amount of money.

If the price is right, we can move on to the rest of our questions. If not, then we might come back to you for another project when we have more money.


  • Where is your company based?
  • Does a development team use developers that are located in different offices? Or will a development team always be located in the same office?
  • Will your developers work USA hours? What is the overlap with the East Coast/West Coast of the USA?
  • Can your developers attend online dev team meetings a few times per week?
  • How fluent in English are your developers? How do you manage the issue that we get a great developer who is not fluent in English?

The choice of location is important for two main reasons: collaboration and cost. 

You need to decide if you are going to want to be in direct contact with the developers of your application, and whether the outsourcing company will even allow that kind of access. If you have a small development team, you (or your CTO) are going to want to have regular development meetings with the team. The team will feel more connected with your product and may do a better job if they were to see and hear the person who is actually the brains behind the product. You can use the development meetings to clear up any issues and to clarify what you expect in case there are any misunderstandings.

If you are located in California, and you hire a team in India, then you will hardly overlap with the working hours of the developers and you may find it difficult to schedule regular team meetings. However, if you are on the West Coast and get a team that is based in Latin America, then it will be much easier.

You might be able to benefit from a country whose currency is weak against your own currency. You want to make sure that you stay around that $35-$45/hour figure, and you might be able to strike a better deal if you can pay a company in their local currency, even after the currency conversion costs. Even if the development shop quotes their prices in US dollars, you might be able to negotiate a better price if the US dollar is strong against the other currency.


  • Are the developers hired as full-time employees of your company, or do you hire on an as-needed basis?
    • If you hire on an as-needed basis, do you use local talent, or do you go to platforms such as Upwork?
  • How many developers do you have, and what is the (technological) composition of your staff?
  • How much turnover do you have?
    • If a developer is in the middle of a project for a client, and that developer leaves, how do you determine how to backfill?
    • If we hire a mid-level developer, and they leave, and the only person on your bench is a senior developer, would you backfill with the senior developer and charge the client the rate that they were paying for a mid-level developer?
    • If a developer leaves, how much downtime could the project expect?
    • What does your bench look like?
  • How do you hire your developers? What does the screening process look like?
    • How do you determine the various levels of developers? (Senior, mid, junior)
  • Is a developer assigned to multiple clients or to just one?

I want my clients to have the perception that their remote development team is no different than having an onsite team of full-time employees. This means that the team is stable, is stocked with the skill set that we need, experiences very little turnover and that all parts of the team are working as one cohesive unit.

Ideally, I want the same developers to stay with the project during the full lifecycle. I expect a developer to work a full day on my project, and I don’t want that developer to multitask between different clients. I want the developer to be totally dedicated to what I am building.

I like to use remote development companies that have relatively low turnover. In the case of turnover, I want to see the departing developer do knowledge transfer, and for the new developer to take over in a seamless fashion. To get a sense of what it is like to work for the remote development company, I might check websites like Glassdoor in order to see what the developers are saying about their company.


  • Do you have UX experts on your staff? 
    • Could we see examples of UXs that they have designed?
  • How do your developers keep up to date with the latest technology?
  • How do you determine what new skillsets to invest in and support?
  • How do you build and maintain centers of domain expertise?

At the beginning of this article, I talked about the various roles that I may need for a project. I want to make sure that the remote development shop has a wide variety of skills available on an a la carte basis, even if I don’t need those skills right now.

Most importantly, I want to make sure that the application that I build is not only visually pleasing but has a workflow that makes logical sense So, I might ask to bring on a UX or UI expert for a little while during the project.

Company Reputation

  • What makes your development company different than the others?
  • If I were to go to a review site like Clutch, what would I expect people to say about your company?
  • If you deliver a product which is buggy, what are your policies around make-goods?
  • How many customers have you had? How many do you currently have?
  • Can we get references from your past customers?
    • Can we talk to former customers?
  • We would like to retain all IP rights for what you have developed for us. Do you have any issues with that?
  • We would like total secrecy surrounding our startup. Will you agree to not tell your potential customers about us?

There is nothing like hiring a company that has delivered successful products with a minimum of fuss. To that end, try to find out as much about that company as possible. Look on sites like Clutch and Glassdoor for comments about the company. Try to join some startup-oriented Slack channels and see what others have to say. If the company has a USA-based sales office, try checking with their local Better Business Bureau.

Will the company take pride in their work? If so, they might offer some type of guarantee on the quality of their work. You may want to see if they will fix bugs for no cost for a certain period after the delivery of the MVP.

The Software Development Process

  • Do your developers have experience in developing an application like the one we are giving you?
  • Can we interview each developer before they start?
  • Are there any tools that we need to provide your developers?
    • Do you provide your own licenses for your internal development needs?
  • How much technical input do you need from your client in order to start developing?
  • Will your client have direct access to the developers?
    • If not, then does your client have to pay for a Project Manager?
    • What is your preferred method of collaboration?
    • Does a client absolutely need a project manager from your company? Can we manage the project by ourselves?
  • How do you ensure that a developer does not create key-person risk and use a technology that is relatively little-known?
  • Do you do code and architecture reviews of your developer’s projects to make sure that they are maintaining sufficient quality?
  • If the client has a CTO or Chief Architect, can they do the code reviews? And what is your policy if they find major problems?
  • How do your developers document their work?
  • Do we receive the source code once you are done?
  • We would like the developers to use our Github repo. Is there any problem with that?
  • What development practices do your developers like to use?
    • What development methodology do your developers prefer? (Scrum, Kanban, etc)
  • How do your developers fit into the software lifecycle (do they push all the way to live, or does that come back to our company)?
  • Who is responsible for infrastructure (design, changes, maintenance)?

Now that you have determined that the cost is affordable, the location lines up with your time zone, the development shop is reputable and can supply the necessary skillset that you need to develop your application, you now need to determine how you will be interacting with them on a day to day basis.

It helps to have a development team that is familiar with the domain that your app will cover. For example, if you want a stock-trading application to be written, it would be beneficial to have developers who are familiar with the domain (ie: what is a stock, what is a trade, what is an order, connectivity to an exchange or to a clearinghouse), and with the technologies that will be used to implement the application (ie: MongoDB, real-time messaging, WebSockets, etc). If the remote development shop does not have developers available with the necessary knowledge, how long will it take the development shop to recruit that talent? If training is required, are you responsible for the training, or is that an expense that the development shop will bear.

Will you have the opportunity to interview a pool of developers (or, at least, look at their resumes) for your team? Or will the development shop give you the team? Sometimes it might be best to let the dev shop handle the choosing of the team because there might be existing synergies in place. It might be a group of people who have already worked together to build successful products. At the very least, I would want to look at the resumes of each of the developers.

Software licensing costs can be huge. Because most applications are developed on the cloud using PaaS (Platform as a Service) and SaaS (Software as a Service), your licensing costs for commercial software can be part of the cloud vendor’s charge. But what if you had a specialty piece of software that you need to install on your cloud-based server? Do you need to purchase a license for every developer?

You should have direct contact with the developers on your team. Be wary of any development shops that insist that your only interface with the developers is through a paid project manager. You also want to have weekly meetings with the entire development team, using a collaboration method like Skype, Google Hangouts, or Zoom. You may want to invite the developers to a special Slack channel that can be used for real-time collaboration.

The source code should be available at all times through a cloud-hosted source control repository, such as GitHub, GitLab, or BitBucket. All code should be well documented in English, and the code should be unit tested.

Some remote development companies will offer the free services of a CIO or senior-level Architect to ensure that the code and architecture are of high quality. Even though some startups consider their MVPs to be throwaway efforts, I like to preserve as much code as possible.

The list of questions that this article contains is only a small portion of the questions that a trained senior technologist will look for when evaluating a remote development company. CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. Please consider CTO as a Service to be your senior technical advisor on any projects that you might develop.

Marc Adler

CTO as a Service

September 2019

Thanks to Andrew Cherry for reading an early version of this document.

The AWS Messaging Stack – SQS, SNS, Kinesis

Executive Summary

There are two main use cases for considering messaging in most applications – lightweight eventing and messaging between components, and processing large streams of data. If you are using AWS as your main platform, it makes sense to consider using AWS-native platforms to handle the messaging. For the case of simple peer-to-peer messaging, where the exact order of the messages is not mandated and a single application has to be notified, SQS is a good solution. It is easy to set up and manage, and it can be used to perform a simple event notification or be used in a request/response paradigm. For the case where a number of applications have to be notified of an event, SNS is a good solution, as it supports the concept of topics. SNS can be used to deliver notifications to a wide variety of notification mechanisms (email, SMS, Lamba, REST), and can interface with SQS queues for notifications to individual applications. If your application has a use-case where it needs a heavy amount of streaming, then Kinesis is an easy-to-use streaming platform for processing large streams of data. Kinesis Analytics can be used to perform Complex Event Processing, and to find interesting events within the streaming data. For what most applications need, Kinesis is preferable to Apache Kafka because it is easy to set up and administer, and you will avoid many of the operational complexities that usually plagues a deployment of Kafka.

SQS (Simple Queuing Service)

SQS provides the most basic way to perform interprocess communication – a simple queue. With SQS, you simply send a piece of data (a Message) into the tail end of the queue. One or more consumers will read messages from the front of the queue. If there are multiple consumers reading from the same queue, then any one of these consumers can receive the message that is at the front of the queue. The message is delivered once and only once to a single consumer.

SQS comes with two different kinds of queue. The Standard queue imposes no ordering of messages. The Fifo queue, which is only available in certain AWS regions, will guarantee ordering of the messages within the queue.


There are two different was that an SQS consumer can poll for messages. The first way is short polling. In this scenario, a consumer will look at the queue, read any messages that are in the queue, and then will return. If there are no messages in the queue, then the short-polling consumer will return immediately. The other way is to do long polling. In this scenario, the consumer will wait for a number of seconds for messages to appear in the queue.

Since AWS charges you for each SQS request, it may be more economical for your consumer to do long polling, since there will be fewer requests if messages are put into the queue on an infrequent basis.


A Message contains a payload, which can be any amount of data up to 256K bytes. A message can also contain custom attributes, which are name/value pairs. When a message is placed into a queue, it can have a non-zero Time-to-Live (TTL). If the message has sat in the queue without being consumed, and its TTL has expired, then the message is automatically deleted from the queue. 

If you want to have messages up to 2GB in size, AWS has a Java-based library that uses S3 as the message storage.


You can have Cloudwatch monitor certain metrics of a queue and automatically scale out by adding additional instances.

aws autoscaling put-scaling-policy –policy-name my-sqs-scaleout-policy -–auto-scaling-group-name my-asg –scaling-adjustment 1 –adjustment-type ChangeInCapacity


The JMS way of programming is available for SQS. Amazon distributes a library called the Amazon SQS Java Messaging Library, and it supports using SQS as the JMS messaging provider. However, it is only available if you are programming in Java, which leaves the C# and NodeJS developers out in the cold.

SNS (Simple Notification Service)

SNS is a way to send a message to a topic and then route the message to a number of notification mechanisms. The message can be routed simultaneously to one or more destinations. The destinations include:

  • HTTP REST endpoint
  • AWS Lambda Function
  • SQS queue
  • Email
  • SMS

Unlike SQS, SNS does not have a dead-letter queue where it routes undeliverable messages. SNS is basically a fire-and-forget mechanism.

SNS messages are pushed to the destinations. The destination consumer does not have to worry about polling for messages.

In the SBS world, the RUN Event Dispatcher (RED) has some of the same functionality as SNS.

SNS-SQS Integration

You will notice that one of the delivery mechanisms that SNS supports is to push a message into an SQS queue. On the other side of the queue could be an application that can read the message and take some action. 

You can even push the single message to multiple SQS queues in order to execute some tasks in parallel.


IMPORTANT NOTE- Kinesis Streams is not available for the AWS Free Tier

Kinesis is the preferred hosted streaming platform for AWS. It differs from SQS and SNS in that Kinesis feels comfortable ingesting continuous streams of data, such as a stream of real-time stock quotes or a stream of signals from millions of IoT devices.

A Kinesis stream is subdivided into shards. Each shard can process a stream of data in isolation of other shards. This provides a degree of load-balancing. Each piece of data can contain a “partition key”, which directs that piece of data to be processed by a specific shard. 

Each Kinesis consumer has a shard iterator which is used to read data from the stream. In this sense, Kinesis is similar to Kafka. Since data is persisted in the stream, a consumer can retrieve data from the beginning of the stream. This supports the concept of “late joiners”, in which a new subscriber can retrieve all of the events that they might have missed. A side benefit of this is that it is easy to replay data for various testing scenarios.

Consumers run of EC2 instances. You can auto-scale consumers by hooking Kinesis up to Cloudwatch, and adding additional EC2 instances dynamically when needed.

Kinesis Analytics

AWS has a service that works with Kinesis that allows you to perform queries on the data in the stream as that data passes through the stream. This service is called Kinesis Analytics, and it gives Kinesis the kind of Complex Event Processing (CEP) capabilities that systems like Streambase and Esper have.

Kinesis Analytics uses a dialect of SQL to perform processing. You can use this capability to detect certain conditions and generate events, or use can use this capability to enrich or transform the data.

The streaming SQL code below detects a condition where the change in a stock price is over 1%. If this condition is detected, an event is generated and put into another stream. A consumer of the other side of this new event stream can send a message to a user or trigger some sort of algorithmically-based trade. 


    (ticker_symbol VARCHAR(4), sector VARCHAR(12), change DOUBLE, price DOUBLE);



      SELECT STREAM ticker_symbol, sector, change, price 


      WHERE  (ABS(Change / (Price – Change)) * 100) > 1;

Apache Kafka

An alternative to the AWS-hosted messaging systems mentioned above is to provision your own EC2 servers and install and run Apache Kafka on those servers. 

I will not talk about the technology around Kafka here, as this has been discussed elsewhere. But I will talk about the differences between using Kafka on AWS and using one of the native AWS platforms.

Several articles point to Kafka being more performant that Kinesis for very high-throughput use cases. But if your application does not have the amount of streaming data that would compel you to use Kafka, then Kinesis is a simpler platform to use.

Some Pros for Kinesis

  • Managed service
  • Removes operational headaches and costs
    • Tuning Kafka can be a challenge, and Kafka engineers are difficult to find
  • Costs can be lower than Kafka for a similar environment
    • With Kafka, you need hardware for the instances, for Zookeeper, for replication, and for data storage for retained messages
  • Fits into the rest of the AWS stack seamlessly
  • Consolidated monitoring via AWS CloudWatch
  • Elasticity – we can bring up Kinesis when we need it.
  • Scale-out transparently at times of heavy usage
  • Kinesis Analytics add-on
  • We avoid the fragile nature of the integration of Zookeeper and Kafka

Some Pros for Kafka

  • No vendor lock-in
  • Wide support for C# clients
    • Most Kinesis APIs are Java-based
  • We do not have to pay for Kafka, but we would have to pay for the EC2 servers that host Kafka
  • Kafka SQL gives Kafka some of the same capabilities as Kinesis Analytics
  • Since Kafka is open-source and part of the Apache project, we have visibility into Kafka (bug fixes, roadmap)
  • Supports wildcard subscriptions
  • Integrated with other Apache projects like Spark, Storm, and Samza

As a happy medium between performance and a full-managed service, we can consider using a completely managed Kafka service that runs on AWS. This service is run by Confluent, who is a consultancy that specialized in Kafka and was founded by the original Kafka developers at LinkedIn.



Native AWS ServiceYYYN
Chargeback modelPer-request plus data egressPushes and deliveries. Different pricing for different delivery methods. Shards per hour N/A
Push vs PullPullPush  
Max message size256K 1 MBConfigurable, but defaults to 1MB
Max message throughputUnlimited for standard queue300 tps for FIFO 1000 PUT records per second per shard1 MB/sec input and 2 MB/sec output per shard 
Message delivered to multiple consumers?N YY 
Message order preserved?Only in FIFO queues NY 
Durable messages?Messages are stored on multiple servers Y 
Replay of messages?NNYY
Data retention60 seconds to 14 days1-14 days (if not deleted)1-7 days
Wildcard subscriptions?N   
Max queue depth120K in-flight messages20K for FIFO queues  N/A N/A
Dead-letter Queues?Y N 
Encryption?Y Y 
ScalingTransparently auto-scale through CloudwatchTransparently auto-scale through CloudwatchYou can increase the number of shards used but you need to pre-provision the shards 
Monitoring through Cloudwatch?Y Y Y 

Language support for APIsC#, C++, Java, NodeJS, Python, Ruby, PHP, GoC#, C++, Java, NodeJS, Python, Ruby, PHP, GoC#, C++, Java, NodeJS, Python, Ruby, PHP, GoC/C++, Python, Go, Erlang, .NET, Clojure, Ruby, NodeJS, Perl, PHP, Rust, Java, Scala, Clojure, Swift

Comparison of Kafka vs Kinesis

Storage of MessagesAs much as you want. On the cloud, you pay for storage.24 hours by default. Up to 7 days with a config change
Ordering of MessagesPartition levelShard level
Message Delivery SemanticsKafka guarantees at-least-once delivery by default. Kafka supports exactly-once delivery in Kafka StreamsKinesis Data Streams has at least once semantics
ReplicationUse Confluence’s MirrorMaker to replicate a topicAll message automatically replicates to all 3 availability zones
ScalingAdd more partitions to a topicAPI call to increase the number of shards
Partition/Shard ModificationIncrease only and does not repartition existing dataRe-shard by merging or splitting shards
Partition/Shard LimitationNo limit. Optimal partitions depend on the use case500 shards in US East (N. Virginia), US West (Oregon), and EU (Ireland) regions. 200 shards in all other regions.
SecurityEither SSL or SASL and authentication of connections to Kafka Brokers from clients; authentication of connections from brokers to ZooKeeper; data encryption with SSL/TLSData can be secured at-rest by using server-side encryption and AWS KMS master keys on sensitive data within KDS. Access data privately via your Amazon Virtual Private Cloud (VPC)
ToolsKafka Connect – gets data in and out of Kafka.

Kafka Streams – stream processing of the data that flows through Kafka.
Video Streams

Data Streams

Data Firehose

Data Analytics
MonitoringYammer Metrics for metrics reporting in the serverAWS CloudWatch and CloudTrail
LimitationsDefault is 1MB per message, but can be changed
Message size is limited to 1MB.

You can only get records 5 times per second and up to 2MB per shard.

Need to add more shards (at an additional cost per shard) to scale.
PricingPer node

No concept of elasticity
Per shard hour
Per storage after 24 hours
Per PUT request

Shards are elastic, so you can decrease shards during slow periods

A Startup Founder’s Guide to Smells in Outsourced App Development

In my past roles of CTO and Chief Architect, I have been involved in scores of code and architecture reviews. One of the services that I offer as part of my CTO-as-a-Service consultancy is to perform code architecture reviews of the apps that my clients are trying to go to market with. Some of the code has been developed by in-house staff. But, more often than not, I am called in to review code that has been developed by offshore development shops, the kind of shops that employ reams of “commodity” developers.

One of the reasons why I started CTO as a Service is because people used to come up to me and ask me how they turn their ideas into a product. Because of the time constraints of my full-time jobs, I would offer a little bit of advice in exchange for a beer. I would usually point these people to a number of development shops that I know, hoping that the development shop would be able to take the idea and turn it into a proper app. After that initial meeting, I did not involve myself any longer with the development of the app. I didn’t concern myself with the architecture that the development shop came up with, whether they were using a cloud provider, whether the code was clean, etc.

Startup founders who are non-technical usually have a little bit of money to develop the MVP of an application. Ideally, they would like to have a technical co-founder who would develop the app in exchange for equity, but there are a lot more ideas than there are CTOs with available time who will work solely for equity. So, the startup founder will search Upwork for a remote developer or will gamble with an off-shore development shop that has junior-level resources available for $25/hour. The startup founder usually has some wireframes and a written description of what the application is supposed to do. They throw it over the fence to the remote developers and wait with bated breath for the MVP to be delivered at some time in the future.

The problem that I constantly see is that there is nobody technical who is sitting on the side of the startup founder, representing the interests of the founder to the developers. There is nobody who is sketching out the architecture of the application. There is nobody making sure that the setup on AWS or Azure or Heroku will not incur massive cost overruns. There is nobody who is doing code reviews to make sure that the developers know what they are doing. 

This issue is not confined to startup founders. There are well-established small companies who decide, for some reason or another, that they would like to create an app. For example, a venerable law firm that decides that they want to provide legal advice to their clients on a mobile app, or an old company that specializes in tutoring that decides that they want to offer an electronic version of their test-prep methodology. These companies often do not have anyone technical on their side to interface with the remote development shop.

A List of Code and Architecture Smells

You hopefully go to your doctor every year for a check-up. And, sometimes, you take your car into the mechanic for a yearly tune-up where the mechanic will see if the car is in good shape.

Similarly, you can take your app’s code to an experienced IT professional to see if it is well-designed and resistant to bugs. During this process, the IT professional will examine the code for certain “smells”.

A “code smell” can be defined as:

“Smells are certain structures in the code that indicate a violation of fundamental design principles and negatively impact design quality.”

Someone with many years of experience developing systems (such as myself) can take a look at a code base or an architecture, and instinctively detect if there are funny smells around it.

The list of code and architecture smells comes from real-life reviews that I have done over the years. I will be adding to this list as I do more reviews for my clients. The list will never contain any mentions of my clients, my past employers, any specific products, or any development shops.

This list is designed for non-technical startup founders. As such, I will explain each smell and why it is bad for your app.

Code Smells

Too Many Hands in the Code

Someone’s coding style is like their fingerprint. When you give your app to a remote development shop to work on, you have no idea how many different people are going to write the code. Transitioning between different sets of developers takes time, and knowledge transfer can be spotty. Too many hands in the code might mean that the development shop has a lot of turnovers, it could mean that they are taking developers and transitioning them to higher-paying projects, or it could mean that they want to get your app done as quickly as possible and are attacking your app in a highly parallel fashion.

Lack of error checking and exception handling

There is nothing worse than having your app crash all of the time. If an app is unreliable, users will reticent to use it. Have you ever experienced the “spinner of death” in an application, where the app seems to be stuck? Nothing will drive users away faster than having their computers or phones lock up.

Every function call should be checked for null arguments, bad values, null return values, and other unexpected situations. Edge cases should be tested (ie: a negative number is used where a positive number is expected). A consistent exception-handling policy should be implemented.

Lack of Comments and Documentation

During the lifetime of your application, the source code will pass through many hands. Remote development shops can rotate different developers in and out of your code base. Although the lack of comments is not strictly a “code smell”, the lack of comments will result in a greater transition time for new developers to learn your application’s code.

At the very least, there should be comments for every module or class, and there should be comments for every public function and property.

Comments will not only assist developers in learning a codebase; it can also be used to generate system documentation. There are tools that will scan the source code and generate various types of documentation. 

If your application has APIs that are meant to be used by other third-party developers, the API documentation can be automatically generated. The API documentation should adhere to the OpenAPI (aka Swagger) specification. Once the API documentation is in OpenAPI format, it can be presented in an easy-to-read format on a web page.

In addition to the comments in the code, all architectural decisions should ideally be memorialized. A Wiki, such as Confluence, is ideal for keeping records of design decisions. You should insist that your remote development shop delivers to you documentation around all major design decisions, and what alternatives were considered and discarded.

Copying of Code

Development companies who are under time pressure to implement apps can find themselves inserting copies of code into multiple places within the application. There can be multiple problems associated with copy-copying. First, if a bug is found in that piece of code, it needs to be fixed in multiple places. Second, the code can possibly “leak” responsibilities. 

A good design will establish a firm “division of responsibilities” between different parts of the code. For example, there might be only one place in the code that is responsible for debiting and crediting a customer’s bank account. If you copy that code into different places within an app’s codebase, then the responsibility of debiting and credit a user’s account will have “leaked” into other parts of the code, making the app more difficult to maintain.

Breaking the Separation of Concerns

The code of an application should be composed of different layers. The typical layers include User Interface, Services, Repository/Persistence, Models, Controllers, Framework, Communications/Messaging, etc. Each layer has a specific responsibility. This rule is called “separation of concerns”.

A developer should not let responsibilities leak from one layer into another. I have seen code where a specific vendor’s user-interface library was referenced in the Persistence Layer. This not only creates a tight binding between the User Interface and the Persistence Layer, but it makes it more difficult to switch to another User Interface toolkit.

Another way of doing “separation of concerns” is to divide the application into various microservices. During an architecture review, we might want to consider the effort and the benefits of refactoring the codebase into microservices.

Improper Class Hierarchies

Well-designed code is akin to beautiful poetry. One thing that we look for in a code review is a sensible hierarchy of classes, and well as adherence to well-known object-oriented techniques. Common functionality might be put into a base class, which other classes inherit from.

A common base class for business objects is useful in order to implement common functionality such as validation, flagging if a model has been changed, shared properties (such as ids and audit information), etc.

Single Points of Failure

One of the most important things that we look for in an architecture evaluation is the single points of failure within an application. If a single service fails, or the connection to an important external service fails, will the entire application be hosed?

Your developers have no control over third-party systems that your application depends on. But you need to see if there are other third-party systems that can provide the same information that your application can use as a secondary source.

If your application is broken up into microservices, then there are various high-availability patterns that your developers can use in order to make your application more resilient to failures. 

Key Person Risk

You want to avoid situations where a single developer on the remote development team is the only person who knows a key technology that your application is built upon. Likewise, any knowledge about “tricky” or “complex” parts of the codebase or the architecture should not lie in the hands of a single person. If it is, then you have “key person risk”. The consequences are that if this person leaves the company, you may not be able to fix errors or make improvements in the app.

It’s important to memorialize all important architectural decisions. A wiki such as Confluence is good for capturing all of the information about the architecture and the development process.

Resume-Oriented Development

Sometimes, the only reason why a developer chooses a certain technology is that they want to put that technology on their resume, no matter if that technology is not particularly right for the application. A choice of a nascent technology by a single developer could possibly result in key man risk, especially if that developer decides to leave the team.

In particular, I have seen many occurrences of a developer choosing the wrong database technology simply because the developer wants to gain experience with NoSQL databases at the expense of the client.

It is more difficult to back an application out of resume-oriented development. The best time to catch this is at the design stage. An experienced architect will be able to evaluate various choices for certain technology and identify if any of those technologies could present a key man risk.

Building when you should Buying

The decision to create your own software from scratch vs buying an existing product is always a difficult one. 

When you incorporate someone else’s product into your architecture, you are beholden to the whims of that company to fix bugs and to release new features that you might need. You might also have the issue of “vendor lock-in”, where it is impossible to move away from a vendor’s product. On the positive side, you save money and effort by having your developers use something that is fully-baked.

During the design phase of a product, an experienced architect can recommend existing third-party products and frameworks that you can use within your application in order to get faster time-to-market without an excessive amount of risk.

One of the advantages of using a cloud platform like AWS or Azure is that there are new platforms coming out all the time, and these platforms are fully supported by cloud vendors. Almost every need that an app has an XXX-as-a-Service solution.

There is also a world of open source software that can be leveraged. There are certain rules to follow when choosing open source frameworks, but that is the topic of another article.

SDLC Smells

The Software Development Lifecycle (SDLC) describes how software is supposed to be developed. The chart below shows all of the steps that comprise a proper SDLC practice. In reality, many remote development shops do not follow each and every step, mainly out of concerns about cost and time.

Lack of Unit Tests and Integration Tests

Developers are changing code all of the time, fixing bugs and adding new features. As they are creating new capabilities for your app, you have to feel comfortable that the existing code will continue to work. 

This is where unit testing and integration testing come in. A unit test will test certain functionality in a piece of code. Ideally, every function within a module (or class) should have an associated unit test. Ideally, every code path in the application should be tested by a unit test suite. The percentage of all code paths that have unit tests is called “code coverage”. Ideally, an application should have 100% code coverage, but unless your project is developed using Test Driven Development (TDD), that percentage often falls short of 100%.

It’s extremely important to not only test the “happy path”, but to test that the code will not break when it encounters bad data. This means that you need to write unit tests that pass bad data into functions. You also need to be able to cause the code to generate an exception and write tests to make sure that the code is generating these exceptions when encountering error conditions.

Of course, you need to make sure that the code actually has error checking and exception handling, which is a huge code small if it doesn’t.

An integration test will test the interaction between multiple modules. For example, a function that is supposed to debit a bank account should result in a decrease in the customer’s “available balance” column in the database. So, not only do you need to make sure that the “DebitCustomer()” function is tested, but you need to read the database to make sure that the available amount has changed.

A regression test will compare the results of two separate runs of the test suite to make sure that the values produced by running a test suite are not different (or different within a certain tolerance) than the results of the previous run of the test suite. In addition to comparing actual values that are produced by the app, we can also test the performance time of various parts of the app. We want to make sure that a new version of the code is not noticeably slower than the previous version. If the app starts to crawl, then users might get frustrated and might abandon the app.

There are tests that can be written to test what happens to your system when it is under heavy load, which is what will happen when Oprah Winfrey mentions your app in an interview. These are called soak tests and stress tests.

No Continuous Integration

Continuous Integration is important in the Software Development Life Cycle. When a developer checks the code into a source code repository (like Github or BitBucket), all of the unit tests are automatically run. A failing unit or integration test will identify problems immediately before those problems seep deep into the app and find their way into the production version of the app.

You might have several developers working on your app. For example, in many development shops, you would have one developer working on the backend (the database and the server), you would have one developer working on the front-end (maybe a website), you might have one developer working on the iPhone version of the app, and finally, you may have another developer working on the Android version.

All of these developers would have the code “checked out” from the source code repository, maybe in a separate branch. When it comes time to release a new version of the app, all of the developers would have to merge their code back into the main codebase. The longer a developer has a branch checked out, the more prone the app is to errors when the developers check in their code. If things don’t go smoothly, as they often don’t, it can result in what is known as “integration hell”. This costs you time and money.

Continuous Integration should be performed frequently, at least once per day, if not more. The mantra is “Check in early, check in often”.

Some remote development shops do not use continuous integration. Why is this? Writing unit tests and integration tests are tedious and costly. Development shops make money on churning out as many apps as possible in a given timeframe. When you do not have testing and continuous integration set up, you incur “technical debt”. And, sadly, tech debt always comes back to bite you.

Data Access Smells

No Caching

In order to access and store your app’s data, you have to make a call into a database. However, database calls are relatively expensive. There is the time it takes for the data to be transferred over the network. Certain database operations can take a relatively long time to execute. And there can be “deadlocks” that occur when multiple callers try to access the same data.

One of the common architectural smells is the absence of caching. There can be no better performance-killer than hammering a database with a lot of operations, especially the operation of putting data into the database. 

Good application design will employ “caching”. With caching, data is stored in memory within the application. The cache is checked for the data your app needs, and only if that data is not found will the database be accessed.

I can’t tell you how many code reviews I have done where I have detected an absence of caching. And the introduction of even a small in-memory cache has resulted in dramatic improvements in the performance of certain parts of an app.

Multiple Avenues To Update a Database

When we use caching, it’s important to keep the cache in sync with the database. Imagine if the cache contains a value that is the customer’s available balance, and some other app goes directly to the database and updates available balance in the database. The cache will not know about this update, and as a consequence, the original app might have a “stale” value for the customer’s balance.

During an architecture review, it is important to look for all code, services, and applications that can change a database, and if possible, we need to force all database access to go through a single gateway. This way, we can ensure that any caches that the app maintains will be totally in sync with the values in the database.

If this is impossible to do, then there should be a mechanism where database-update events are broadcast to various services in the app, so that these services know that they need to refresh part or all of their cache.

Bad Use of ORMs

An Object Relational Mapping framework (ORM for short), is a way that developers can translate between high-level objects and a relational database. If a developer writes the app in an object-oriented language like JavaScript, C#, or Java, it can be tedious to deconstruct an object and store each field in appropriate tables in a database. An ORM will translate that object into SQL calls to the database.

The problem sometimes with ORMs is that the SQL code that is generated can be slow. This has been a long-running complaint with ORMs.

Careful examination should be made of the database interaction that is controlled by the ORM. There are various tools for databases like SQL Server that will monitor the calls to the database and may suggest improvements. You should consider moving frequently-used ORM-generated calls into SQL stored procedures, and calling those stored procedures directly.

Data Type Mismatches Between Database and Code

There are applications who store certain data as a non-optimal data type within the database. For example, dates and currency values should not be stored as text fields. This makes it easy to store incorrectly-formatted values inside of the database. Storing dates and numbers as text strings make it more difficult to do arithmetic on the values. The developer first has to convert the text field into the correct data type, hope that the conversion succeeds, then perform the arithmetic on the new value, then convert the result back into a text field. These needless conversions are tedious and error-prone.

Special care also needs to be taken to make sure that the code will handle “nullable columns” correctly. These are values that are optional within the database. The code should never assume that a nullable column contains a valid value.

Out-of-Sync Views

If there are multiple people working on the same case, and there is a change that one person makes, the other person will not see that change until a fresh call to the database is made. That can lead to situations where there is stale data on somebody’s screen.

Many web applications use real-time messaging to notify the user interface that something in the model has changed. For ASP.NET MVC, there are frameworks like SignalR that will help implement real-time notifications between modules of an application.

Security Smells

Storing PII in Plaintext

Personal Identifiable Information (PII) should never be stored in plaintext. I have seen cases where configuration files have sensitive password information that can easily be compromised by a hacker. A full scan of the codebase should be made to ensure that no passwords are stored in plaintext anywhere.

Test data should never include any PII. I have seen cases where a test database contains social security numbers and password in plaintext form.

Passwords and PII should never be stored in a source code repository like Github. It is common to stored passwords in the .env file in Node js applications. Be sure that the .env file is never stored in a repo.

If you are in Europe, all apps have to be GDPR compliant. If it is not, it can mean big fines and possible shutting down of the app. So take PII very seriously.

No Authentication in the public API

Exposing an API layer for your app is a great way of encouraging third-party developers to create new extensions for your app, thereby making your app even more powerful. But, you do not want to make your app the “Wild West”, with unfettered access to your platform.

Make sure that APIs that your app exposes have proper authentication. You may want to make some APIs truly free and public, but anything that writes data to your database or changes the state of your application should have proper authentication around it.

Performance Smells

Frequent Polling of Services

An application can have a  connection to various internal and external services that contain data that is critical to the app. For example, your real-estate app might be connected to an external Multiple Listing Service (MLS) server that contains information about new homes that came on the market. What your application may do is to connect to the MLS service every few minutes, download data, see if any of that data changed since the last time the app connected, and notify the user that something changes.

If there are multiple internal and external services that our app has to connect to and “poll” for changes, then our application can take a performance hit, especially if there are a lot of services to poll.

A much better way of getting data updates is to let the external services “push” data to your application. Your app basically subscribes to updates, sits back, and lets all of the services push events when something interesting happens.

Applications should attempt to migrate from the “pull” model to a more modern “push” model of updates. This is usually accomplished by using webhooks, which are HTTP-based POST calls to a URL when an interesting event occurs.

No Provision for Scaling

My favorite thing to say to startup founders is: “What happens if Oprah mentions your product? What happens if you have 10,000 people hitting your app servers at the same time? Can your app and infrastructure handle the load, or will your app crash and burn?”

A good architecture will be able to let an app scale up seamlessly. Cloud providers like AWS and Microsoft Azure and Google Cloud Platform provides services that will automatically let you scale your application upwards when you need it, and downwards during slow times.

An architecture review, along with proper soak and stress testing, will make a startup founder more comfortable that their application will be able to handle the “Oprah Moment”.

The list of various smells that this article contains is only a small portion of the smells and anti-patterns that a trained senior technologist will look for when evaluating the architecture and codebase of applications. CTO as a Service has over 30 years of writing systems, leading development teams, and doing architecture reviews. Please consider CTO as a Service to give your applications a health-check from time to time.

Marc Adler

CTO as a Service

July 2019

Writing a Redis Module on the Mac

You can extend the functionality of your Redis 4.x installation by writing custom modules in C using the Redis Module SDK. Since Redis 4.x is only available on Unix-based systems, you need to write your Redis modules on a Unix-like system such as MacOS and use compilers like gcc. (Redis for Windows is only supported up until Redis 3.2.) Your Redis module must be a Unix shared library. This shared library can be loaded into Redis when Redis is first started or can be loaded dynamically into an already-running instance of Redis.

I have attempted to document the process of writing a Redis module using gcc and using Visual Studio Code as my development environment. The example shown below comes right out of the Redis Module SDK.

Note that the Redis Module SDK is still under development. For example, it does not yet have an API that supports SET-based functions.


Make sure that the Gnu gcc compiler is installed on the Mac. Open up a terminal and just enter the command


Open Microsoft’s Visual Studio Code. It’s helpful to install the official Microsoft C/C++ extension. 

Download the Source

Clone the Git repo for the Redis Module SDK. The main Github site is here. In a Terminal window, navigate to the directory where you want the Git repo to be downloaded to. Then enter the command

git clone

Modify the Source

After the source code is downloaded, edit the file rmutil/sds.h and change line 82 to

#define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (struct sdshdr##T*)((s)-(sizeof(struct sdshdr##T)));

(Change the “void*” to “struct sdshdr##T*” in order to silence the Mac’s gcc compiler)

Build the Source and the Example Module

In the Terminal, go to the root directory of the Redis Module SDK, and just enter the command


This will build the single library (librmutil.a) that you need to link your custom modules with. It also builds the example that comes with the Redis Module SDK. It will also build the shared library ( that is the custom module that you will load into Redis.

Using Visual Studio Code

Run Visual Studio Code. Open the main directory that the Module SDK is in. We need to create JSON-based configuration files that tell Visual Studio Code how to build the application and how to run/debug the application. These configuration files go into the .vscode subdirectory under your project.

The tasks.json file will tell Visual Studio Code how to run the make command.

To run the example, you need to launch the command

/usr/local/bin/redis-4.0.6/bin/redis-server –loadmodule ./


 "version": "0.2.0",
 "configurations": [
     "name": "(lldb) Launch",
     "type": "cppdbg",
     "request": "launch",
     "program": "/usr/local/bin/redis-4.0.6/bin/redis-server",
     "args": ["--loadmodule", "./"],
     "stopAtEntry": false,
     "cwd": "${workspaceFolder}",
     "environment": [],
     "externalConsole": true,
     "MIMode": "lldb"


 "version": "0.1.0",
 "command": "make",
 "isShellCommand": true,
 "tasks": [
         "taskName": "Makefile",

         // Make this the default build command.
         "isBuildCommand": true,

         // Show the output window only if unrecognized errors occur.
         "showOutput": "always",

         // No args
         "args": ["all"],

         // Use the standard less compilation problem matcher.
         "problemMatcher": {
             "owner": "cpp",
             "fileLocation": ["relative", "${workspaceRoot}"],
             "pattern": {
                 "regexp": "^(.*):(\\d+):(\\d+):\\s+(warning|error):\\s+(.*)$",
                 "file": 1,
                 "line": 2,
                 "column": 3,
                 "severity": 4,
                 "message": 5

Running the Module

In Visual Studio Code, run the debugger. This will launch a copy of Redis with your new module loaded. You can put breakpoints into your module’s code and watch Redis execute the module.

While the debugger is running a copy of Redis, open up a Terminal and run the redis-cli program. In redis-cli, enter the commands:> EXAMPLE.HGETSET foo bar baz
(nil)> EXAMPLE.HGETSET foo bar vaz
(integer) 7> EXAMPLE.PARSE PROD 5 2
(integer) 10> EXAMPLE.TEST

Creating a Slackbot on AWS using Golang – Part 3 – AWS Lambda Functions

Marc Adler

CTO as a Service

In the previous article of the series, we created a Quote Alerter for Slack and AWS. The Quote Alerter will notify a user on Slack if the price of a stock went above or below a certain target price. The Golang-based code runs on AWS and uses a Postgres database on RDS in order to store all of the alert subscriptions and the list of current stock prices.

In this article, we will migrate the quote-checking logic to an AWS Lambda function.

(There is an article on the CTO as a Service blog that discusses using Lambda with Visual Studio Code and C#/.NET. That article has some good intro material on Lambda functions on AWS, so you are encouraged to glance over it if you have any basic questions about Lambda.)

Why migrate the Slack Stock Bot to Lambda functions? Mainly for illustrative purposes for this series of articles. In reality, there might be some relatively time-consuming business logic that you might want to take out of the main code path of an application and run asynchronously with a Lambda function. In the domain of equities and quotes, you might want to have a separate serverless function that will compute some Greeks and either store those values in a database, or enrich our Slack notification messages with those Greek values (like the delta and gamma).

In the migration path that we are going to undertake, we will just start off with a simple Go-based “Hello World” lambda, and then slowly drag in the parts of the Slack Stock Bot that we need in order to implement the alert mechanism.

The source code to this project can be found here

Overview of the Migration

  1. Create a new Lambda Function using the AWS Lambda dashboard
  2. Create a new CloudWatch trigger that will cause the new Lambda Function to run
  3. Create a very simple Go-based Lambda using the Go/AWS SDK, and test it out
  4. Change the existing Slack Stock Bot code so that we can import packages from it easily
  5. Change the code to the new Lambda Function so that it replaces the ticker-based price break checking
  6. Deploy the new Lambda function
  7. Test the function by manually firing the CloudWatch event

Creating a New Lambda Function on AWS

The first step in the process is to create a new Lambda Function by using the AWS Lambda dashboard.

After clicking on the Create Function button, you will be presented with a form that you need to fill out with the information about your new function.

We call our new function priceBreachChecker. We make sure that the function has the Golang runtime installed, and we will use a previous execution role that we have set up. The execution role determines which AWS services the lambda function can access.

After creating the lambda function, we need to specify what kind of events will trigger the execution of the function.

Creating the CloudWatch Trigger

If you recall, the current Slack Stock Bot code creates an application-based ticker that will check for price breaches at certain intervals. This uses the Golang ticker package. At every tick, the code will call the quote service to retrieve all the current prices for the stock symbols that have alerts on them. It will then call some SQL that will ask the Postgres database to see which current prices have breached the price limits that were set up.

In order to simulate this ticker, we will use AWS CloudWatch events. You can set up CloudWatch to call a Lambda Function at regular intervals or on a cron-based schedule (ie: every weekday at 12:00 PM and at 5:00 PM).

Back in the Lambda dashboard, add a new trigger. Choose CloudWatch Events from the list of triggers on the left side of the page.

Now we need to set up the interval that the CloudWatch trigger will fire.

Click on the Add button. For the Rule Type, choose Schedule Expression, and use rate(60 minutes) as the expression. This will tell CloudWatch to fire the event every hour.

Click on the Add button. You will get confirmation that the new Lambda Function has been created.

Before we look at the CloudWatch dashboard, notice that there is a way that you can upload a ZIP file of your Go-based Lambda code. We will not be using this. Instead, we will be using the AWS CLI from within Visual Studio Code to deploy our code.

Changing the CloudWatch Trigger

Let’s look at the CloudWatch dashboard in order to verify that we have a trigger. On the left side of the page, find the Events / Rules menu item and click on it.

If you click on the name of the rule, you can see some further details.

Under the Actions button, choose Edit. You will see that there is a rule that controls the interval that the event will be fired. If you want to change the interval at which the Price Breach Checker will run, then adjust this interval.

You can also have this rule trigger additional Lambda functions. Let’s say that we have a separate price-fetching Lambda function for every different quote service we support. We can have this CloudWatch rule trigger each of the separate Lambda functions. If you want to do this, choose a new Lambda function, and click on the Add Target button.

Creating a Simple Go-based Lambda

The main docs on Lambda and Go can be found here:

We need to download and install the AWS Lambda SDK for Go

go get

Now let’s get busy with Visual Studio Code. We are going to create a new folder for our new Lambda function.

Creating Tasks for Visual Studio Code

We can create a list of tasks that Visual Studio Code will run to do the build and deploy of the Lambda function. In Visual Studio Code, go to Terminal / Configure Tasks, and edit the tasks.json file.

My tasks.json file looks like this:

   "version": "2.0.0",
   "tasks": [
           "label": "Build",
           "type": "shell",
           "command": "go",
           "args": [ "build", "-o", "priceBreachChecker"],
           "options": {
               "env": {
                   "GOOS": "linux",
                   "GOARCH": "amd64"
           "group": {
               "kind": "build",
               "isDefault": true
           "label": "Zip",
           "command": "zip",
           "args": [ "", "priceBreachChecker", "appSettings.json"],
           "dependsOn":[ "Build" ]
           "label": "CreateAndDeploy",
           "command": "aws",
           "type": "shell",
           "args": [
               "lambda", "create-function",
               "--function-name", "priceBreachChecker",
               "--region",  "us-east-2",
               "--profile", "default",
               "--role", "arn:aws:iam::901643335044:role/service-role/woof_garden_canary",
               "--handler", "priceBreachChecker",
               "--runtime", "go1.x",
               "--zip-file", "fileb://./"
           "options": {
           "problemMatcher": [],
           "dependsOn":[ "Zip" ]
           "label": "UpdateAndDeploy",
           "command": "aws",
           "type": "shell",
           "args": [
               "lambda", "update-function-code",
               "--function-name", "priceBreachChecker",
               "--region",  "us-east-2",
               "--profile", "default",
               "--zip-file", "fileb://./"
           "options": {
           "problemMatcher": [],
           "dependsOn":[ "Zip" ]

There are four tasks here.

One is the Build task, which compiles the Go code. Notice that we use two special environment variables that tell the Golang compiler about the platform that the code should be generated for.

“GOOS”: “linux”,
“GOARCH”: “amd64”

The AWS server that runs your Lambda function is a runs Amazon’s own version of Linux and runs Go code that is compiled to the amd64 chipset.

The next task will Zip up the compiled executable and the appSettings.json configuration file. AWS requires that a ZIP file contains the assets for your Lambda function. Notice that the Zip task has a dependency on the Build task, so if you run the Zip task, it will automatically do the build as well.

(Note: Instead of using a separate application settings file, you can set environment variables in the Lambda function dashboard, and then read those environment variables.)

The CreateAndDeploy task will not be used here, because we already created the Lambda function using the AWS Lambda dashboard.

The final task is UpdateAndDeploy. This is used to update AWS with new versions of the code. It will upload the ZIP file that was created by the Zip task. We made the UpdateAndDeploy task dependent on the Zip task so that the build. zip and upload processes can be done with a single command.

Writing a Sample Lambda Function in Go

We will create a simple Go package which will just echo the arguments to the log.

Here is the code:


package main

import (

    ev ""

func main() {

func priceBreachChecker(ctx context.Context, event ev.CloudWatchEvent) (int, error) {
    lambdaContext, _ := lambdacontext.FromContext(ctx)
    log.Println(fmt.Sprintf("In priceBreachChecker handler: context is %+v", lambdaContext))
    log.Println(fmt.Sprintf("In priceBreachChecker handler: event is %+v", event))
    return 0, nil

Notice the arguments for the priceBreachChecker function. There are several different function signatures for the entry point, and somehow, the AWS Lambda runtime is able to figure out how to marshal the various triggers to the functions. The CloudWatchEvent is the struct that contains all of the information that the CloudWatch trigger generates.

Testing the Lambda

The first step to testing out this simple Lambda function is to build it, zip it up, and deploy it to AWS. To do this, I ran the Zip and the UpdateAndDeploy tasks from within Visual Studio Code.

I went into the CloudWatch Event Rules and temporarily changed the trigger interval to 30 seconds.

Then I went into the CloudWatch logs and waited until the trigger fired. Here is what the log looked like:

Success!!! The two log messages that the function generated can be seen in the CloudWatch log.

(Don’t forget to change the trigger back to 60 minutes, or else your Lambda function will run every 30 seconds)

Packaging the Slack Stock Bot

Most programming environments support the use of packages. In the world of C#, we use NuGet to import third-party packages. In the Node.js world, people use npm, and Java, most developers use Maven.

When we write our new price-checking Lambda function, we would like to import the code from our existing Slack Stock Bot. We have seen that you can import packages from Github using the go get command.

Since our existing code is already up on Github, let’s import it:

go get

Easy enough, right? But look at the various error messages that Go Get gives us. These error messages all look like this:

       local import "../configuration" in non-local package

What does this mean?

In the file stockbot.go, we have a bunch of imports that look like this:

import config "../configuration"

It seems that Go Get does not like any relative references in the code that it imports. By “relative reference”, we mean an import whose directory is relative to any other directory. These references usually start with the dot character, like “../” or “./”.

So what do we need to do? We need to find all relative references in the import statements in our code, and turn them into references into our Github repository.

import ""

You can read more about this issue here.

Now that we have fixed all of these references, and we have checked the code back into Github, we can now run the command

go get

Merging the Slash Command Code in with the Lambda

We will import the parts of the Slack Stock Bot package that we need.

When the Lambda function is loaded, the init() function is called. This is a feature of Go. The init function is the place where you can do one-time initialization.

In the init() function, we read the configuration information (we will need the webhook part of the appSettings), we create the Stockbot (which is the interface to the quote services), and we will create the AlertsManager (which does the checking for the price breaches).

When the Lambda function is triggered, we call the function to check for the price breaches, and for every breach, we notify the user through Slack.

We insert a number of logging statements, just so we can trace the running of the code.

package main

import (

    ev ""

    config ""

var theBot *stockbot.Stockbot
var theAlertManager *alerts.AlertManager
var appSettings *config.AppSettings

func init() {
    // Put any one-time initialization code here
    configMgr := new(config.ConfigManager)
    appSettings = configMgr.Config()

    theBot = stockbot.CreateStockbot()
    // defer theBot.Close()

    // Create the AlertManager
    theAlertManager = alerts.CreateAlertManager(theBot)
    // defer theAlertManager.Dispose()

func main() {

func priceBreachChecker(ctx context.Context, event ev.CloudWatchEvent) (int, error) {
    lambdaContext, _ := lambdacontext.FromContext(ctx)
    log.Printf("In priceBreachChecker handler: context is %+v\n", lambdaContext)
    log.Printf("In priceBreachChecker handler: event is %+v\n", event)


    return 0, nil

// checkForPriceBreaches - checks for price breaches
func checkForPriceBreaches() {
    fmt.Println("checkForPriceBreaches: Checking for price breaches at " + time.Now().String())

    theAlertManager.CheckForPriceBreaches(theBot, func(notification alerts.PriceBreachNotification) {
        log.Println("The notification to Slack is:")
        outputText := fmt.Sprintf("%s has gone %s the target price of %3.2f. The current price is %3.2f.\n",
            notification.Symbol, notification.Direction, notification.TargetPrice, notification.CurrentPrice)

                               notification.SlackUserName, notification.Channel, outputText, appSettings)

    fmt.Printf("checkForPriceBreaches: Finished checking for price breaches at %s\n", time.Now().String())

That’s all we need to do for the new Lambda function. We are ready to deploy and test the code.

Deploying the New Lambda Function

Run the UpdateAndDeploy task from Visual Studio Code. You will see this output:

    "FunctionName": "priceBreachChecker", 
    "LastModified": "2019-06-20T13:14:25.885+0000", 
    "RevisionId": "7278be22-93ab-4bee-8c85-b4fea3a9857e", 
    "MemorySize": 512, 
    "Version": "$LATEST", 
    "Role": "arn:aws:iam::XXXXXXXXXXX:role/service-role/woof_garden_canary", 
    "Timeout": 15, 
    "Runtime": "go1.x", 
    "TracingConfig": {
        "Mode": "PassThrough"
    "CodeSha256": "GVzIhBYObNJY4+ENZ78Emr081ApWxJPOS3KAD/AMbA4=", 
    "Description": "", 
    "VpcConfig": {
        "SubnetIds": [], 
        "VpcId": "", 
        "SecurityGroupIds": []
    "CodeSize": 4848883, 
    "FunctionArn": "arn:aws:lambda:us-east-2:XXXXXXXXXX:function:priceBreachChecker", 
    "Handler": "priceBreachChecker"

This confirms that the new version of the code has been uploaded.

Testing the Lambda Function

In the Lambda Console, create a new test event.

Since our priceBreachChecker lambda function reacts to a CloudWatch trigger, we choose an Event Template that mimics a CloudWatch event.

After you click on the Create button to create the event, go back into the Lambda console and click on the Save button.

Now that the test event has been created and saved, click on the Test button in order to manually fire a CloudWatch event. You should see a Slack notification generated in the log.

Success!!! We successfully created a Lambda function that does the alerting on price breaches. And the notifications show up in Slack too.

All of the source code to this article can be found here:

As always comments are welcome.

Possible Enhancements

Currently, the Lambda function runs and just notifies the Slack user when a price breach occurs. We can enhance the code to compute other values, and output those values to other AWS services. In the last article, we talked about the computation of Greeks. We can send those Greek values to an SNS topic, we can store them in DynamoDb, or we can feed them into a Kinesis stream. We can do this by calling other parts of the AWS-Go SDK from within the Lambda function.

Once you have a Lambda function running inside of AWS, the possibilities are many.

About Me

Marc Adler is the founder of CTO as a Service, a consultancy that provides senior-level technical services to companies who are in need of a CTO or Chief Architect on a “pay for what you use” basis. He was formerly the Chief Architect of companies like Citigroup, MetLife, ADP, and Quantifi. He likes to get himself in trouble with his CIOs by insisting on coding.

Creating a Slackbot on AWS using Golang – Part 2 – Price-based Alerting


In the previous article, I talked about how to create a Slack Slash Command that would return the current price of a stock. So, you could enter the command /quote MSFT into a Slack message field and it would return the current price of Microsoft stock.

The Golang-based server was first run locally and then migrated to AWS using Elastic Beanstalk.

The article ended with a list of features that I would like to eventually implemented in my little Go/AWS/Slack application. This article, and subsequent articles, will focus on developing some of these features.

For this article, I wanted to implement an alerting feature in the Stockbot. With this feature, someone could enter a target price of a stock and be alerted when the current price of the stock went above or fell below the target. Maybe AMZN stock fell below $1000 a share and you want to rush to your financial advisor and buy a share or two?

This new feature requires creating a database which will be used to hold both the alerting subscriptions and the current prices of all of the stocks that all users are interested in. This will let us introduce how to set up a database in AWS and how to talk to that database from a Go application.

A New Branch

Let’s go to our Git repository for a second. We would like to create a feature branch for the new alerting feature. So let’s create a new branch on our local machine.

You can create a new branch through the command line

git checkout -b alerting

or the branch can be created from inside Visual Studio Code:

Business Requirements – Designing the New Slash Command

The requirements of the new command are simple.

A user will tell the Stockbot that they want to be notified asynchronously through Slack whenever the current price of a stock goes above or below a certain price target.

The Stockbot will poll the quote service at regularly-scheduled intervals and will retrieve the current prices of all of the stocks that users want to be alerted on. Whenever a price breaches the alerting price, the Stockbot will send a message to the user.

By default, the user will be notified in Slack by a Direct Message (DM). The user can also choose to be notified through a specific channel. Usually, that channel is a private channel that the user has set up, just for price alerts, but it can also be a public channel.

As far as additional user interactions go, we would also like a way to list all of the alerts that a user has, a way to delete a specific alert, and a way to delete all alerts.

When the user creates an alert, we want to make sure that the symbol is a valid stock. If not, an error should be returned. If the user already has an alert for this symbol, the alert will be updated with the new price target (and possibly with the new direction).

Given these requirements, we can design the new slash command.

/quote-alert [symbol price [below]] [symbol delete] [deleteall] [#channel]


/quote-alertlists all of the alerts you have
/quote-alert HELPprints a help message
/quote-alert MSFT 130sends an alert when Microsoft stock reaches $130
/quote-alert MSFT 130 #myalertssends an alert to the #myalert channel when MSFT stock reaches $130
/quote-alert MSFT 130 BELOWsends an alert when Microsoft stock goes below $130
/quote-alert MSFT deleteremoves the existing alert on MSFT stock that you have subscribed to
/quote-alert deletealldeletes all alerts that you have

The Alerts Database

Given these requirements, we can design the schema for a database that will hold the subscriptions. The database can also hold current prices.

Amazon’s RDS service gives the developer a choice of several different databases to create. For this exercise, let’s choose Postgres since it is one of the databases available on the RDS Free Tier.

Every alert should have at least the following properties:

  • A unique id
  • The id of the Slack user that created the alert
  • The symbol of the stock that the user wants to monitor
  • The target price of the stock
  • The “direction” of the check (above or below the price)
  • The Slack channel that the user wants to be notified in
    • If the channel is empty, then the user should be sent a direct message through Slack
  • An indication that tells us whether this alert has been triggered
    • In case the sending of the alert is slow, we don’t want alerts to pile up

We also would like a simple table that holds the current price for each symbol that has an alert on it.

Let’s look at the SQL that will be used to create the database. Since we will be creating a Postgres database, the SQL below has the Postgres dialect.

create type slackstockbot.direction as enum ('ABOVE', 'BELOW');

alter type slackstockbot.direction owner to magmasystems;

create table slackstockbot.alertsubscription
  id serial not null
     constraint alertsubscription_pk
        primary key,
  slackuser varchar(128) not null,
  symbol varchar(16) not null,
  targetprice double precision not null,
  wasnotified boolean default false,
  direction slackstockbot.direction default 'ABOVE'::slackstockbot.direction,
  channel varchar(128) default ''::character varying not null

alter table slackstockbot.alertsubscription owner to magmasystems;

create unique index alertsubscription_id_uindex
  on slackstockbot.alertsubscription (id);

create table slackstockbot.stockprice
  symbol varchar(32) not null,
  price double precision not null,
  time timestamp

alter table slackstockbot.stockprice owner to magmasystems;

create index stockprice_symbol_index
  on slackstockbot.stockprice (symbol);

In addition to the two tables shows above, we may want to think of having a table with administrative info, such as the time that the last polling was done, the frequency of the polling, and the name of the quote service to pull from. We will leave this for a future exercise.

Finding Price Breaches using SQL

We can join the AlertSubscriptions with the current prices and find all rows that have prices that are either above or below the price target.

SELECT a.slackuser, a.webhook, a.symbol, a.targetprice, a.direction, p.price
  FROM slackstockbot.alertsubscription a, slackstockbot.stockprice p
  WHERE a.wasnotified = false AND a.symbol = p.symbol AND p.price > 0 AND
     ( (a.direction = 'ABOVE' AND p.price >= a.targetprice) OR 
       (a.direction = 'BELOW' AND p.price <= a.targetprice) )

Creating the Database in AWS

If you recall from the previous article, we created a development environment for the Slack Stock Bot on Elastic Beanstalk.

If you click on the green box, you will see the dashboard for the SlackStockBot environment.

In the side panel, click on Configuration. Then scroll down until you see a panel for the Database. You will notice that it is empty.

After you click on the Modify link, you will see a list of databases that are associated with this environment. Click the Create Database button.

You will be presented with a list of database engines. If you are looking to save money, make sure that you check the box at the bottom which will only present you with options that are eligible for the RDS Free Tier. We will choose Postgres.

Name the database and pick the authentication credentials.

In the Network and Security section, I like to make the database publicly accessible so that I can administer the database from my local machine using tools like DbVisualizer or DataGrip.

After the database is created, I will use something like DataGrip to create the tables using the SQL that was shown above.

In addition to setting up this Postgres database in AWS, I also set up a local version of the database for local testing. If you recall from the first article, we used localtunnel in order to have Slack interact with a local version of the Slack Stock Bot.

Progress So Far

We designed the API for the new /quote-alert command. We also created a database and created the two tables that will hold the alert subscriptions and the local prices.

The next stage is to create a new Slash Command in Slack and hook it up to the new version of our server. Then we will write the Golang code which implements the AlertManager.

Adding the New Slash Command to Slack

In the previous article, we saw how to add a new Slash Command to Slack. Let’s do the same thing again. We will create the new /quote-alert slash command.

Once the Slash Command has been created, we need to give it permission to send a message to a user directly and to a specific channel. Click on the OAuth & Permissions link on the side panel.

Then pull down the dropdown under Select Permission Scopes and choose the two permissions.

Click on the Save Changes button.

Now that the permissions have been granted to perform certain actions, we need to set up two Webhooks for the communication. First, enable Webhooks in your command.

In the side panel, click on Incoming Webhooks, and make sure that the webhooks are activated.

Scroll down a bit and create two new webhooks.

At the end of this process, you should have two Webhooks, one for posting to a channel and one for sending a message to a user.

By default, a /quote-alert will send a price alert directly to the user with one of the webhooks. If you enter the command

/quote-alert MSFT 130 #myalerts

then the alert will be sent to the #myalerts channel, using the other webhook.

Modifying the Golang Source

Now that all of the environmental stuff has been set up, we can write some Golang code.

The Source code is located here (alerting branch)

In order to access the Postgres database, we use the pq package. You need to install this package from, and then reference it within the application.

go get

Major Changes to the Code

There have been several things added to the version of the Slack Stock Bot that was developed in the previous article. We are not going to cover each change in this article. But, at a high level, those changes include:

  • The introduction of environment-specific configuration files (appSettings[.env].json), plus a configuration manager
  • A logging manager
  • A Slack Messaging package that encapsulates all interactions with Slack
  • An AlertManager that encapsulates all of the price breach alerting logic
  • Integration with Postgres (either local or RDS)

Changes to the Configuration File

A new Database section has been added to the appSettings.json file. This contains the standard database connection information that will be used to connect to Postgres. There are two new fields for the webhooks that the alerting mechanism will use to send messages back to Slack. Finally, there is the quoteInterval, which is the number of seconds that will elapse between price checks. Bear in mind that the free quote services will limit the number of quotes that you can request per day, so you do not want your price checker running too frequently.

   "apiKeys": {
       "quandl": "[Your Quandl API Key]",
       "worldtrading": "[Your World Trading Data API Key]",
       "alphavantage": "[Your AlphaVantage API Key]"
   "driver": "alphavantage",
   "slackSecret": [Your Slack App's Secret Key]",
   "webhook": "[Your Webhook for Channels]",
   "dmwebhook": "[Your Webhook for direct messaging]",
   "port": 5000,
   "database": {
       "host": "",
       "port": 5432,
       "dbname": "slackstockbot",
       "user":  "[Your database user name]",
       "password": "[Your database password]",
       "SSL": true
   "quoteCheckInterval": 600,
   "disablePriceBreachChecking": false

Polling for Price Breaches

In application.go, a Ticker is created using an interval which is set in the appSettings.json configuration file. Every time the ticker elapses, a function is called to check the prices.

// Create a ticker that will continually check for a price breach
if !appSettings.DisablePriceBreachChecking {
    priceBreachCheckingTicker = time.NewTicker(time.Duration(appSettings.QuoteCheckInterval) * time.Second)
    defer priceBreachCheckingTicker.Stop()

    // Every time the ticker elapses, we check for a price breach
    go func() {
        for range priceBreachCheckingTicker.C {

The responsibility for the price checks is in the AlertManager. We pass a callback function that the AlertManager calls for every price breach. This callback will create an informative message and will post it to Slack using a webhook.

// onPriceBreachTickerElapsed - This gets called every time the Price Breach Ticker ticks
func onPriceBreachTickerElapsed() {
    theAlertManager.CheckForPriceBreaches(theBot, func(notification alerts.PriceBreachNotification) {
        outputText := fmt.Sprintf("%s has gone %s the target price of %3.2f. The current price is %3.2f.\n",
            notification.Symbol, notification.Direction, notification.TargetPrice, notification.CurrentPrice)
        postSlackNotification(notification, outputText)

The check for price breaches works like this:

  • Get a list of all of the stocks that have alerts on them
  • Call the quote service to get the current prices for all of the stocks
  • Save the prices to the database
  • Use SQL to check for price breaches. The SQL code for the check is shown at the start of this article.
  • For each alert that was triggered, set a flag that “logically deletes” the alert so that we do not check again.
    • We can enhance the /quote-alert command so that an alert can be reset
  • Call the passed-in callback function, which is responsible for alerting Slack.
// CheckForPriceBreaches - gets called by the application at periodic intervals to check for price breaches
func (alertManager *AlertManager) CheckForPriceBreaches(stockbot *stockbot.Stockbot, callback func(PriceBreachNotification)) {
    // Get the latest quotes
    prices := alertManager.GetQuotesForAlerts(stockbot)
    if prices == nil {

    // Save the prices to the database

    // Check for any price breaches
    notifications := alertManager.GetPriceBreaches()

    // Go through all of the price breaches and notify the Slack user
    for _, notification := range notifications {
        // Set the wasNotified field to TRUE on the alert

        // Do the notification to slack synchronously

A Word About Architecture and Strategy

By this time, you must be wondering why we used a SQL-based function to detect price breaches, especially if we were ever going to support real-time streaming quotes. After all, making calls to the database is costly in terms of performance, latency, and (in the case of RDS), monetary cost.

Wouldn’t we be much better off using some in-memory collections? For example, we could use a map where the keys are the list of stocks that have active alerts, and each value could be a sorted collection of alerts.

One of the reasons that I chose the database-centric way of doing the comparison is just so that I could find a way to introduce a database in this series of articles. I wanted to give the reader exposure to uses databases both in Golang and in an Elastic Beanstalk environment.

If we wanted to be architecturally flexible, we could introduce a Strategy Pattern. We could have a strategy for database-based comparisons and a different strategy for memory-based comparisons.

We can implement the Strategy Pattern by using a factory to create the quote comparator, and we can assign the comparator to a field within the AlertManager struct.

type QuoteComparator interface {
    findPriceBreaches(alerts AlertMap, currentQuotes []QuoteInfo)

type AlertManager struct {
    . . .
    quoteComparator QuoteComparator
    . . .

func createAlertManager() {
    . . . 
    alertManager.quoteComparator = quoteComparatorFactory("memory")
    . . . 

func quoteComparatorFactory(strategy string) (comparator QuoteComparator, errs error) {
    switch strategy {
    case "database":
        return &amp;DatabaseQuoteComparator{}, nil
    case "memory":
        return &amp;MemoryQuoteComparator{}, nil
        return nil, errors.New("the strategy cannot be found")

Deploying to Elastic Beanstalk

We need to change the Buildfile so that it fetches the pq library for Postgres, and so that the app’s configuration file is located in the same directory as the binary. The new Buildfile is:

go get
go get
go build -o bin/application application.go
cp ./appSettings.json bin

An important thing to note is that, by default, Go applications on Elastic Beanstalk use Port 5000. If you change the port from within the configuration file, then you should also tell Slack that the Stock Bot command uses the new port.

Another thing that we might want to consider is, instead of using RDS, using Docker and deploying our own Postgres database. Elastic Beanstalk fully supports setting up Go applications using Docker. We can leave Docker to a future article.

Testing the new Slash Command

Let’s put in an alert for Johnson and Johnson’s stock.

/quote-alert JNJ 140.0

We see that the Slack Stock Bot works

That message looks a lot nicer than just printing out plain old text. Slack allows you to format output in different ways.

attachment := slack.Attachment{
    Color:    "good",
    Fallback: "You successfully posted by Incoming Webhook URL!",
    Text: outputText,
    //Footer:        "slack api",
    //FooterIcon:    "",
    Ts: json.Number(strconv.FormatInt(time.Now().Unix(), 10)),

msg := slack.WebhookMessage{
    Attachments: []slack.Attachment{attachment},
    Username:    slackUserName,
    Channel:     slackChannel,
slack.PostWebhook(getWebhook(slackChannel, appSettings), &amp;msg)

Slack also supports something called Blocks, which allow more complex formatting and options for the user to interact with your message. Conceivably, we can use Blocks to present a “Buy” or “Sell” button, which would generate an order to the user’s financial advisor.

Merging the Alerting Branch Into Master

Now that we are done implementing the alerting feature, we can merge the alerting branch back into the master.

Go to the Github repository.

Click on the green button that is labeled Compare & pull request.

Type in some comments and then click on Create pull request.

You will see that there are no merge conflicts. Click on the Merge pull request button.

Confirm the merge

You will get the confirmation that everything was merged successfully, Now you can pull the new branch to your local machine.


In this article, we enhanced the original Slack Stock Bot code so that the user could subscribe to alerts. The alerts were stored in an AWS RDS database, and we used a SQL-based strategy to detect any price breaches. We came up with a new Slash Command called /quote-alert which allows a user to create or delete a price breach alert. Finally, we deployed the new code to Elastic Beanstalk and successfully tested it out.

In the next article, we will make a few more enhancements. One of the things that I am thinking of is making the price comparison into an AWS Lambda function. We can also use the new Slack Messaging to implement a simple workflow. We should start putting in unit tests, and we can start taking advantage of AWS CodeBuild and CodeDeploy.

Stay tuned for the next article.


Trouble Connecting to the Postgres Database from Elastic Beanstalk

If you find that you are having problems connecting the Slack Stock Bot to RDS, go into the EC2 instance that hosts the database and change the Incoming Connection rules.

  1. Go into the RDS dashboard and find your database. Then click on the name of your database.
  2. In the dashboard for your database, go to the Security Group Rules section, and find the Security Group that is associated with Inbound connections. Click on that.
  3. In the Security Group, look at the Inbound tab. Make sure that port 5432 (Postgres) is open to your application.