🐹 Mole - Workday dependency search with Go

What problem does this project solve?

Workday is an ever-expanding ERP that is most widely known for it's HR offerings. In Workday, there's no way to quickly find what objects are dependent on another object. Over time, it's really easy to make a small change and unexpectedly break something else. There are some reports you can run, but nothing very thorough and so you end up with manually reviewing a subset of objects and then crossing your fingers nothing bad happens downstream as a result of a change.

Here's a few real-world examples where Workday's out-of-the-box tasks and reporting falls short:

• We are acquiring Company B, how do we tell which reports are filtered on Company A and may need to include or exclude Company B?
• We rebranded our Company to "Acme Brand". How do we find all notifications, reports, fields, and integrations mentioning our old name?
• We want to tweak a security group or role in the tenant, what things reference that group and could break as a result of this change?

About the project

This project is named Mole because it "digs up your Workday dependencies." You can read all about what it does here.

The web application is primarily a form where users can enter credentials to connect to their Workday tenant as well as keywords they want to search for. After submitting the form, the user will get back form errors if there are any or a confirmation message that their search has been placed in a queue. Once the search completes, the user gets an email with a list of any Workday objects containing the keywords that were provided in the form.

The CLI application is a very basic wrapper to invoke searches locally on my computer without the overhead of the web application or by referencing a file as a list of keywords.

Key design choices and components

(In no particular order)

• I chose Go for this project over Python (Django) for a few reasons.
• I'm having a lot of fun with Go and wanted a "substantial" project to build and learn.
• It's very unlikely this project will be enhanced or updated frequently, so compiling to a binary on the server with as few dependencies as possible makes deployment really simple.
• Go routines make it easy to handle longer-running background processes. With Python, I'd likely need to use Celery and Redis or some other dependency to broker messages.
• Go web apps allegedly use less server resources than similar Python web apps.
• slog for logging since I heard that's coming into the standard library.
• Go templates for html pages (embedded into binary).
• Background tasks. Long running searches to Workday are passed from the http handler function(s) to a background task to avoid timeout issues. The results are emailed to users after the search is completed.
• Web app middleware:
• Rate limiting across the application.
• CSRF protection via github.com/justinas/nosurf.
• Setting secure headers.
• Panic recovery.
• Web app handles "graceful shutdowns" and will wait 30 seconds or until all background tasks are complete to shut down.
• Web app is deployed on a ~$4/mo AWS VPS.
• TLS certificates and reverse proxy handled by Caddy.
• Systemd for launching and restarting the Go binary.
• A few bash scripts build the binaries and deploy them to the server.
• Security.
• CSRF protection on the form.
• Secure headers like Content-Security-Policy.
• Requires TLS v1.3+ with specific ciphers
• Reasonable timeouts set on the server.
• Form field validation.
• HTTP to HTTPS redirects handled with Caddy.
• Workday has very sensitive data and I don't want myself or anyone else to get into trouble while using this app. No Workday API credentials or API responses are saved on the server in files or in a database.
• Usage or rate limiting.
• Users can only search for a handful of keywords at a time. (Leaving the possibility for a "premium" tier with a larger or unlimited number of keywords)
• The server hosts a few projects and I wanted to avoid saturating all the server resources with this project. I implemented a buffered channel with a worker go routine to limit the number of concurrent requests. The web app will show users a red flash message asking them to try searching again in a few minutes if the limit was reached.
• The app currently relies on Cloudflare managed challenges to avoid bots or other kinds of abuse since that seems to be easier to implement and a better user experience than captcha and I'm not requiring user accounts with verified emails.
• CLI uses the standard library flags package.

Future enhancements

• It needs tests.
• This project is dependent on "private" Workday API's. As far as I know, there is no guarantee that they won't be taken away or limited at some point in the future, which could make this project useless.
• There is no database. This means limited or no analytics, no subscription model, etc.
• There is logging on the server with slog, but no "observability" built into the app..
• The app doesn't send a useful email when it encounters an error connecting to Workday's API to tell the user why they didn't get a response back.
• I'm on the fence about making the repo public and open sourcing this since it could be a product one day.
• There are likely certain authentication policy requirements in Workday that would make this app unusable for some people.