Tech Debt while learning a new language ecosystem. Putting 20 apps into Git.

Updates and open questions about problems Harlan is working to solve and tools he is working with.
harlanji
Posts: 13
Joined: Mon Jul 24, 2023 6:44 pm

Tech Debt while learning a new language ecosystem. Putting 20 apps into Git.

Post by harlanji »

I decided to learn Python when I got a new laptop that doesn't allow running unsigned binaries from the Microsoft Store. Guess how many options that leaves me for development ecosystems? Not many. No NodeJS, no Clojure/Java JVM, no precompiled Python modules. But there are plenty of pure Python modules, so I've made it a long way.

In under a year's time I'd built around 20 apps. I converged on various aspects within the code and structure but over the past year pretty much every app was very different, learning from lessons in the prior app. If it's an app that I use and change frequently I might've modernized it to some degree. But the name of the game has been to run up the tech debt in the name of learning the language and libraries. If I don't like that aspect of the ecosystem then what's the point of investing in learning distribution packaging and beyond?

Similar could be said, what if I don't like the distribution packaging options? I think it's lower risk because of the popularity of the language itself, and ability to use the language to fine tune packaging steps. Blah blah blah, rationalizations. At the end of the day we have limited energy and I wanted to practice on real world applications and there is no option to use Git from the App Store.

Now though, I have a cloud server and confidence in the language and libraries. And it's been so long since I've worked on some of the apps that I want to get them version controlled so I can refactor and add features in a controlled way. I did make snapshots and releases along the way, which I've used to materialize a rough Git history for my own reference and that of any mentees/students or future content writers.

In a sense running up all that debt even helped another aspect of my career development, which is needing to practice a lot of Git and packaging in a short time span--allowing 100% focus and no shortage of problems to address. This isn't unlike the work involved in joining an early stage company and taking over for a departed developer. Businesses optimize for time to market and with a small team there may not be resources to check code into git or manage releases either. If the company is successful and needs to scale its development team then the need arises to add version control and formally manage changes.

So here we are. I have like 20 apps, 5-10 of which I use regularly. Several of which still produce useful reports or have code I'd like to reuse in future apps. A few apps have 2-3+ apps within one directory because they're similar or work on the same data and that's just how I started hacking. Sometimes I split one app into 2, and then one of those again. I've added 7 to Git sofar and have pushed into release packaging territory on one or two and a dependency in the distant past that I can study and apply to each app in succession.

This is what I was up to when I toasted my Raspberry Pi's SD card around March or so. It feels like I'm getting nothing done, but getting it onto a publicly available Git server is actually doing quite a lot.

The big question mark beyond just splitting the apps into steps of history and even just splitting 2-3 related apps from one directory is what to do with the app data, which I've opted not to check into Git. The approach I have in mind is to create Minio/S3 buckets to hold the data and then wire up the apps to work against S3 instead of the file system. That's a lot of work, a quicker option would be to manually S3 sync data, might set it up for that.

I'll probably write more about this process, just adding a journal entry to track where I am now.