Drew De Ponte - A backstage perspective on running a company, building & selling apps | drewdeponte.com

The Git Patch Stack Methodology

2026-01-31T00:00:00+00:00

TL;DR</h2>

Modern Git workflows optimize for features, but best practices optimize for integration, review quality, and history.

The Git Patch Stack Methodology replaces feature-centric pull requests with small, architectural patches stacked directly on top of mainline. Each patch is:

Small and reviewable</li>
Continuously rebased and integrated</li>
Architecturally meaningful</li>
Historically valuable</li> </ul>
Development happens outside-in, while review and integration happen inside-out—one patch at a time. The result is better reviews, fewer conflicts, cleaner history, and faster integration without abandoning pull requests.
A Note on Tooling</h3>
This article describes a methodology, not a tool requirement.
That said, some of the workflows described here—tracking patch identity across rebases, managing patch stacks, and opening pull requests per patch—introduce friction if done manually.
To address that, the Git Patch Stack CLI exists.
👉 https://git-ps.sh</a>
The CLI is designed to support this methodology directly on top of Git and existing platforms like GitHub and Bitbucket. It automates the mechanical work (branching, cherry-picking, tracking patch identity, updating pull requests) so developers can focus on creating high-quality patches and integrating continuously.
The rest of this article explains why the methodology works. Tooling simply makes it practical at scale.
Git Patch Stack Methodology</h2>
To truly understand the Git Patch Stack Methodology, it helps to start with how software changes are typically made.
At a high level, most changes fall into one of two categories:

Hosting my Git Repositories

2026-01-06T00:00:00+00:00

For a long while now it has become more and more evident that you just can't trust companies to host your data. This is true even if you are paying for said services. This has become even more obvious with the introduction of large language models, etc.
Additionally, the products that exist in the space of Git hosting are less than ideal in my opinion. Instead of them focusing on making Git hosting as good as it possibly could be. They are focused on building AI platforms that integrate with all your Git repositories. Which I have zero interest in when I am paying for Git hosting. Beyond that these companies are built on a software development model that is clearly less than ideal but they have no interest in rethinking that model.
For these reasons and more I have decided it probably makes the most sense to simply host all my Git repositories myself.
The following are my notes around how I accomplished this so that if anyone else wants to do the same they would hopefully be able to figure out.
What we need</h2>
Basically, we need a server to host the repositories and run the software. You can of course pay some VPS provider like Digital Ocean or you can run your own server at your house or wherever you want. Note: If you run it at your house you will likely need to figure out some sort of Dynamic DNS. I believe Cloudflare provides this as a service. But I am sure there are others.
Then you need to have an operating system. I personally prefer Arch Linux so I have a server running Arch Linux.
We are going to be hosting our Git repositories over the SSH protocol with the help of a tool called Gitolite</a>. Gitolite</a> facilitates authenticating users and managing access to the Git repositories over SSH public/private key pairs.
The Setup</h2>
This portion assumes you already have an Arch Linux system setup for your server.
Get SSH Daemeon Setup</h3>
First thing we need to do on that server is installed the openssh</code> package so that we will have the ssh daemon installed.
sudo pacman -Sy openssh</code></pre> Next we want to start the ssh daemon.
sudo systemctl start sshd</code></pre> Then we want to enable it so that it starts up at boot.
sudo systemctl enable sshd</code></pre> We can then verify it is running with the following.
systemctl status sshd</code></pre> Then I configured sshd by editing /etc/ssh/sshd_config</code> by updating it so that the following are set. Explanations of what they do are in comments above them all below. This is effectively to harden sshd as may be exposed to the internet directly and it needs to be as secure as possible.
# Enable public key authentication PublickeyAuthentication yes # Tell it where to find the file containing authorized keps AuthorizedKeysFile .ssh/authorized_keys # Prevent password authentication PasswordAuthentication no # Prevent challenge response auth ChallengeResponseAuthentication no # Disable PAM authentication UsePAM no # Prevent root login PermitRootLogin no # Prevent keyboard interactive auth KbdInteractiveAuthentication no # Don't allow any empty passwords PermitEmptyPasswords no # Restrict allowed users AllowUsers youruser gitolite # Disable agent & TCP forwarding unless you need it AllowAgentForwarding no AllowTcpForwarding no X11Forwarding no # Disable tunneling unless needed PermitTunnel no # Disable rhosts-style trust IgnoreRhosts yes HostbasedAuthentication no # Limit auth attempts MaxAuthTries 3 # Limit concurrent connections MaxSessions 2 MaxStartups 10:30:60 # Drop idle connections ClientAliveInterval 300 ClientAliveCountMax 2</code></pre> Then I restarted the sshd to have the changes take place.
sudo systemctl restart sshd</code></pre> After this I simply scp'd my public ssh key over to the server and added it to the authorized_keys</code> file as follows so that I could easily ssh over to the server and manage things.
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys</code></pre> Then I simply made sure that the file had the correct permissions. chmod 600 ~/.ssh/authorized_keys chown -R user:user ~/.ssh</code></pre>Install & Setup Gitolite</h3> First I installed the dependencies as follows. sudo pacman -Sy base-devel git</code></pre> Then I installed Gitolite</a> as follows. sudo pacman -Sy gitolite</code></pre> The above created the gitolite</code> user on the system and setup /var/lib/gitolite</code> as it's home directory. Then simply needed to initialize Gitolite. In order to do this I ran the following. sudo -u gitolite gitolite setup -pk path/to/my/public/ssh/key</code></pre> This setup the gitolite-admin</code> repository as well as a testing</code> repository. The gitolite-admin</code> repository is how you configure and manage Gitolite. So you simply clone it and make changes to the configs in there and commit and push it up. It then handles creating repositories appropriately for you with the configured access controls. Which supports what you need for private repositories as well as anonymous (unauthenticated) repositories which are useful for opensource. What about Off-site Backups</h2> One side benefits I was getting from the services I was leaving was off-site backups. So, I figured I might as well address that in my setup as well. Turns out there is a pretty great backup tool called Restic</a> that integrates with Backblaze B2 that allows you to configure backups to be done and stored in a Backblaze B2 bucket pretty easily. First I had to decide what I wanted to backup. In my case I simply wanted the following. /var/lib/gitolite/repositories /var/lib/gitolite/.gitolite</code></pre> Then I installed Restic</a> as follows. sudo pacman -Sy restic</code></pre> Then I had to go and create my Backblaze B2 bucket and an Application Key, which I captured the bucket name, key id, and key itself and did the following. sudo mkdir -p /etc/restic sudo nvim /etc/restic/gitolite.env</code></pre> I then created filled /etc/restic/gitolite.env</code> with the following content. export RESTIC_REPOSITORY="b2:BUCKET_NAME:/gitolite" export RESTIC_PASSWORD="use-a-long-random-passphrase" export B2_ACCOUNT_ID="your-application-key-id" export B2_ACCOUNT_KEY="your-application-key"</code></pre> Then I set the perms on that directory and file as follows. sudo chmod 600 /etc/restic/gitolite.env sudo chown root:root /etc/restic/gitolite.env</code></pre> Then I initialized the restic repository. sudo -i source /etc/restic/gitolite.env restic init</code></pre> Then I ran a manual backup to test things out. restic backup \ /var/lib/gitolite/repositories \ /var/lib/gitolite/.gitolite</code></pre>restic snapshots</code></pre> Then I added an exclusion file to exclude unneccessary files. nvim /etc/restic/gitolite.exclude</code></pre> and gave it the following content. **/tmp **/*.lock **/lost+found</code></pre> Then I tested manual backup again with it. restic backup \ --exclude-file /etc/restic/gitolite.exclude \ /var/lib/gitolite/repositories \ /var/lib/gitolite/.gitolite</code></pre> Then I came up with a simple retention policy and tested it as follows. restic forget \ --keep-daily 14 \ --keep-weekly 8 \ --keep-monthly 12 \ --prune</code></pre> This removes old snapshots, etc. based on the retention policy. Then I created a backup script so that I could automate this. nvim /usr/local/bin/restic-gitolite-backup.sh</code></pre> with the following content. #!/usr/bin/env bash set -euo pipefail source /etc/restic/gitolite.env restic backup \ --exclude-file /etc/restic/gitolite.exclude \ /var/lib/gitolite/repositories \ /var/lib/gitolite/.gitolite restic forget \ --keep-daily 14 \ --keep-weekly 8 \ --keep-monthly 12 \ --prune </code></pre> Then I made it executable as follows. chmod +x /usr/local/bin/restic-gitolite-backup.sh</code></pre> Then I registered it as a systemd service by creating the following. nvim /etc/systemd/system/restic-gitolite-backup.service</code></pre> with the following content. [Unit] Description=Restic backup of Gitolite repositories Wants=network-online.target After=network-online.target [Service] Type=oneshot ExecStart=/usr/local/bin/restic-gitolite-backup.sh</code></pre> Then I created a systemd timer to run it automatically. nvim /etc/systemd/system/restic-gitolite-backup.timer</code></pre> with the following content. [Unit] Description=Daily Restic Gitolite backup [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target</code></pre> Then I simply enabled it as follows. systemctl daemon-reload systemctl enable --now restic-gitolite-backup.timer</code></pre> And verified that it was properly registered and running with the following. systemctl list-timers</code></pre> Finally I tested the backups to make sure I could restore from them as follows. restic snapshots</code></pre>restic restore latest --target /tmp/restore-test</code></pre> In theory you should be able to restore an individual repo as follows. But I have not tested this yet. restic restore latest \ --include /var/lib/gitolite/repositories/myrepo.git \ --target /tmp/restore-test</code></pre>OpenSource Repositories</h2> Setup hosting of anonymous repositories with git://</code> protocol which doesn't facilitate any authentication. I do this via the git-daemon</code> which comes with git. It also has the benefit of playing nicely with Gitolite</a> and therefore allows sharing the same repositories store. To do this I created a systemd service description for it. /etc/systemd/system/git-daemon.service</code></pre> with the content [Unit] Description=Git Daemon (anonymous read-only) After=network.target [Service] User=gitolite Group=gitolite ExecStart=/usr/lib/git-core/git-daemon \ --reuseaddr \ --base-path=/var/lib/gitolite/repositories \ --disable=receive-pack \ --informative-errors \ --verbose Restart=on-failure [Install] WantedBy=multi-user.target</code></pre> Then I enabled and started the service as follows. sudo systemctl daemon-reload sudo systemctl enable git-daemon sudo systemctl start git-daemon sudo systemctl status git-daemon</code></pre> Then I simply exposed the git-daemon</code> port 9418</code> in the firewall so that externally people would be able to access the opensource repositories. Dynamic DNS</h2> I am using Cloudfare for DNS zone hosting. So I whipped up the following little script ~/bin/update_dns_record_with_public_ip</code>, to dynamically set the IP address. #!/usr/bin/env bash set -euo pipefail ZONE_ID= RECORD_ID= CLOUDFLARE_API_TOKEN= SUB_DOMAIN= TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S') LOGFILE="$HOME/logs/cloudflare-ddns.log" # Fetch the RECORD_ID of the subdomain. This only has to be done initially to get the value for RECORD_ID # So just uncomment this, save the file, and run the script to get the RECORD_ID. Then simply update the # script with the RECORD_ID value, recomment these lines, and save the script and you should be good to go. # # curl -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?name=$SUB_DOMAIN" \ # -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" # exit 0 # Fetch the public IP address using ifconfig.me IP_ADDRESS=$(curl -s ifconfig.me) # Get the IP address Cloudflare currently has for the subdomain DNS_IP=$(curl -s \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \ | jq -r '.result.content') # 3. Compare existing DNS record's IP addr to the current public IP addr if [[ "$IP_ADDRESS" == "$DNS_IP" ]]; then echo "$TIMESTAMP - IP unchanged ($IP_ADDRESS). No update needed." >> "$LOGFILE" exit 0 fi RESPONSE=$(curl -X PUT "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$RECORD_ID" \ -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ -H "Content-Type: application/json" \ --data "{ \"type\": \"A\", \"name\": \"$SUB_DOMAIN\", \"content\": \"$IP_ADDRESS\", \"ttl\": 120, \"proxied\": false }") # Check success SUCCESS=$(echo "$RESPONSE" | jq -r '.success') if [[ "$SUCCESS" == "true" ]]; then echo "$TIMESTAMP - DNS update successful ($IP_ADDRESS)" >> "$LOGFILE" else ERR_MSG=$(echo "$RESPONSE" | jq -r '.errors[]?.message') echo "$TIMESTAMP - DNS update failed: $ERR_MSG" >> "$LOGFILE" fi</code></pre> This script expects your user's hame directory to also have a logs</code> directory to be able to write it's logs out to. This script fetches the public IP address using ifconfig.me and then uses the Cloudflare API to update the subdomain. In order for this to work you need to create an API Key that is setup for Edit zone. This can be done from the Cloudflare -> Avatar -> Profile -> API Keys. Then to use this you obviously would need to fill in the variables at the top of the script. The ZONE_ID can be obtained by selecting the zone in the Cloudflare UI and looking in the right sidebar for the API section and grabbing the zone id value. The RECORD_ID I believe is only available by using the API call that is commented out near the top of the script. Once I had the script setup I was able to test it by just running it. But, I don't want to manually do this. I want it to automatically be run. So I installed cronie</code> with the following. sudo pacman -Sy cronie vi mkdir -p ~/.cache sudo systemctl enable cronie.service sudo systemctl start cronie.service</code></pre> Then I edited my users crontab via crontab -e</code> and gave it the following content. */5 * * * * /home/YOUR-USERNAME/bin/update_dns_record_with_public_ip >> /home/YOUR-USERNAME/logs/cloudflare-ddns-cron.log 2>&1 0 0 * * * truncate -s 0 /home/YOUR-USERNAME/logs/cloudflare-ddns.log</code></pre> The above registers two cron jobs. The first one runs the script every 5 mins to make sure that the DNS record is up to date. The second one runs once every day and truncates the log file so it does not get out of hand. I then waited and checked the log files to make sure everything was working as expected. Don't also forget that you need to likely modify your router configuration to forward port 22 from the outside of your network to your server's internal IP address. That is if you aren't using a hosted VPS. If you are using a hosted VPS you shouldn't need this entire Dynamic DNS section at all as you generally get a static IP address. What about PRs, comments, etc.</h2> Personally, I don't want or need any of those things. There are plenty of tools that can be used to facilitate those types of interactions without the need for the Git hosting to do them. A very basic one is the concept of patches and emails to say the least for open source repositories.
Shower Thoughts 2026-01-01 2026-01-01T00:00:00+00:00 Welcome to my first Shower Thoughts arcticle. This is a series that I came up with while taking a shower this morning. I am not planning on writing an article every day or anything. As I don't necessarily have significant shower thoughts every day. But when I do this is an outlet for me to get them out and share them with my future self and the world. Of course they will primarily be business, product and engineering focused as that is where my career is spent. Coming back to Sharing Things</h2> For about a year now I have taken a step back from sharing things that I am working on or sharing things that I have been thinking about. This was partially because of various things going on in my life, e.g. switch to Principal Engineer at OnDiem, etc. But probably most significantly it was because it takes a decent amount of time and effort to share things and I just wasn't sure the value in terms of what it brings to other people, how much it helps my standing within the community, etc. was there. Now that it has been about a year and I am looking back. I believe I have been looking at the "value" all wrong. I now believe there is something innate to it that helps me stay happy. I am not sure exactly what it is or why. Maybe it is some deep need within me to connect with people in terms of the things that I am working on and thinking about. But I do know at this point it is definitely a significant component that impacts my emotional state. Beyond that if there is one thing I have found to be true over the years. It is that taking the extra few minutes to write something down generally ends up saving me more time in the future. So with those two points in mind I am going to be coming back to sharing things more often. Righting the Wrongs of Software Development</h2> Another topic that came to mind while in the shower this morning is this deep seeded need to right what I see as wrongs within the Software Development world. I have tried to let go of this mental stance over the past year or so as I haven't necessarily seen a direct & clear path to profit from it. I have failed in terms of letting go of this and have decided that for my own sanity, happiness, etc. I really need to double down on this instead of letting go of it. And not worry about the money and the profitability as much. At least right now. So what is this mentality of "Righting the Wrongs of Software Development" you may ask? Well, it is hard to fully articulate. So, I will simply try to call out some wrongs that I see and that I think about, some of which I may be actively exploring addressing. Why is it we can have video games that can perform well above 244 hz while doing tons of computation but we can't have desktop apps that can scroll some basic content smoothly?</li> Why do there have to be so many layers to things in general? It makes everything slower, harder to debug, and much harder to make performant.</li> Why are we trying to build local state based applications in browsers via Javascript? Or maybe even in WebASM if you are really on the bleading edge. It is just more layers of what was already possible in native development in a much better fashion.</li> Why does everyone think the only way to build cross-platform applications is to do so through a browser? Making everything with much more layers and much less performance.</li> Why is it people are so attached to social networks when they generally don't breed true conversation? People would be better offer writing blog posts and simply having people email them if someone wants to comment on a post. Or maybe email a mailing list for that post so that other people interested in that topic could follow along. It would probably be better to facilitate more real meaning conversations. A mailing list could be at the author level or maybe at a per article level. And guess what mailto: links allow you to specify a subject line, body, etc. to start the email with.</li> Why is it every product feels like it has to do a million things. I want the products I use to do less things but do them better. Example I want a simple service to exist that lets people subscribe to mailing lists and that is it. I want another tool that can interact with said service to get emails from that list. Why does everything have to be so needlessly coupled. In Mailchimp for example why can't I send my own custom made raw emails without paying more money?</li> </ul> The list goes on and on and on an on. With tools, services, products in general that I interact with on a daily basis. The hard part is focusing and prioritizing which if any to actual focus effort and energy into. Beyond that, identifying where people went wrong with things and making sure that those things don't happen within my products. Simple Mailing Lists & Flexible Mail Pipeline</h2> This is an idea that I had a while back and I even spent some time working on at one point. But in today's shower thoughts it came back up and makes me think that maybe I should finish out the work to get it at least to an initial phase of completion. Making it fully usable. The core idea here is to split out the service that allows people to subcribe to a mailing list. Making it extremely focused, light weight, performant, cross-platform, etc. Then have another tool that can take Markdown files and produce HTML email content. Then have another tool that can take HTML email content and send it to a collection of users. The idea being that all these components that be simple, focused, performant, and modular. So that they can do their one thing and do it well. This will give me the flexibility. And who knows maybe it will give me some thoughts on how to package things up as a service or a product. But irrespective it will give me the baseline tools to run my mailing list the way I want and give me the flexibility to evolve in the ways that I want. You are probably going. Well, what out click tracking, and open tracking, and all that other stuff that everyone wants their email marketing tools to do. I am not doing any of it. I don't value it. What I am sharing with the world I am going to share if there is 1 person listening or 1 million people listing. That metric isn't useful to me and there are much more valuable and meaningful metrics to pay attention to. This really boils down to simply architecting things in a better way. But this was actually driven out of my frustration with marketing automation tools like Mailchimp, etc. Install multiple PostgreSQL versions from src on Arch 2025-12-12T00:00:00+00:00 On Arch Linux if you need multiple versions of PostgreSQL installed at the same time. Say because you are working on a software product that is currently locked into an older PostgreSQL version while also working on another product that is using a new PostgreSQL version. You are kinda screwed, at least in terms of being able to do it from the packagement management system. They effectively provide the latest version and then also provided through the user packages you can get a previous version to facilitate the upgrade process that PostgreSQL requires. So we have to look for a solution outside of the normal package management system. I tried looking for solutions like a PostgreSQL Version Manager similar in concept to NVM (Node Version Manager) or RVM (Ruby Version Manager), etc. There were a couple that seemed not fully baked and just seemed like a bunch of unnecessary layers of crap. The only alternative I could find was to use Docker which I also didn't want to do as it is a bunch more layers of crap that I didn't want to deal with. We should totally be able to install multiple versions of PostgreSQL side by side on Arch Linux and it shouldn't be a problem. And guess what. I was correct. The following are my notes from when I set this up. The Setup</h2> In the below I specify a custom prefix of /usr/local/pgsql/15.13</code> instead of the default of /usr/local/pgsql</code> because I want to be able to have multiple postgresql versions installed at the same time. Additionally, I specified the openssl-1.1</code> which is an additional package you can install on Arch linux beside the normal openssl 3.6.0, because the pgcrypto</code> extension depends on things that were provided by openssl 1.1 but were removed in openssl 3.x. Beyond that I specified --with-uuid=e2fs</code> because on Arch linux the libuuid</code> is provided and not the ossp-uuid</code> library. I needed to specify this so that when I built the uuid-ossp</code> extension in contrib it could properly build. Out side of that I simply followed the directions in the PostgreSQL documentation with the following configure command to take the above into account. ./configure --prefix=/usr/local/pgsql/15.13 --with-openssl --with-includes=/usr/include/openssl-1.1 --with-libraries=/usr/lib/openssl-1.1 --with-uuid=e2fs make</code></pre> Note: During the process the build would fail. And I would look at why it was failing and it was always simply a dependency that was not installed. I would install the package from the package system and then re-run the build. I did this a handfull of times to get the build to fully succeed. I also followed the same process for the extensions that I installed as well. Start/Stop Versions</h2> You don't really have two versions running at the same time in this setup. So the key becomes how do you start and stop the different versions of PostgreSQL that you have installed. The way I addressed this was simply to whip up some quick shell scripts to start and stop it. For example I created a ~/bin/pgsql_start_15.13</code> and ~/bin/pgsql_stop_15.13</code> to facilitate starting and stopping that specific version of PostgreSQL. The following is the content from ~/bin/pgsql_start_15.13</code>. #!/usr/bin/env sh PGDATA="/usr/local/pgsql/15.13/data" PGCTL="/usr/local/pgsql/15.13/bin/pg_ctl" LOGFILE="$PGDATA/logfile" # go to directory pg_ctl has perms to so that when # pg_ctl tries to return to the directory it doesn't # error. cd /tmp || exit 1 # Check if PostgreSQL is already running if sudo -u postgres $PGCTL -D "$PGDATA" status > /dev/null 2>&1; then echo "PostgreSQL is already running." else echo "Starting PostgreSQL..." sudo -u postgres $PGCTL -D "$PGDATA" -l "$LOGFILE" start echo "PostgreSQL started." fi</code></pre> The following is the content from the ~/bin/pgsql_stop_15.13</code>. #!/usr/bin/env sh # Variables PGDATA="/usr/local/pgsql/15.13/data" PGCTL="/usr/local/pgsql/15.13/bin/pg_ctl" # go to directory pg_ctl has perms to so that when # pg_ctl tries to return to the directory it doesn't # error. cd /tmp || exit 1 # Check if PostgreSQL is running if sudo -u postgres $PGCTL -D "$PGDATA" status > /dev/null 2>&1; then echo "Stopping PostgreSQL..." sudo -u postgres $PGCTL -D "$PGDATA" stop echo "PostgreSQL stopped." else echo "PostgreSQL is not running." fi</code></pre> Fix PostgreSQL Collation version miss-match 2025-11-25T00:00:00+00:00 The following is an example of what you need to do to fix PostgreSQL Collation version miss-match issues on Arch Linux. REINDEX DATABASE dummy; ALTER DATABASE dummy REFRESH COLLATION VERSION;</code></pre> Basically my understanding is that PostgreSQL depends on certain libraries and the version of those libraries are effectively snapshotted when the indexing is done. Hence the COLLATION VERSION. So when libraries are upgraded on the system and PostgreSQL is now linking to a different version of the library. There is no guarantee that it behaves 100% the same as it did with the previous. Therefore, you need to reindex things and then update the COLLATION VERSION. Install Ruby on Arch Linux 2025-11-25T00:00:00+00:00 I had to frum install Buby with the following so that it was setup to find the proper system libs, etc. CFLAGS="-std=c99 -I/usr/include -L/usr/lib -D_GNU_SOURCE" frum install 3.2.2</code></pre> Upgrade PostgreSQL on Arch Linux 2025-11-25T00:00:00+00:00 The following are my notes for upgrading PostgreSQL on Arch Linux when I have installed PostgreSQL through the official package management system. Stop the service with the following. sudo systemctl stop postgresql</code></pre> The PostgreSQL upgrade process which is done with pg_upgrade</code> requires having the previous version of postgres that created the /var/lib/postgres/data</code> directory as the format is locked to major versions. For example you might be upgrading from PostgreSQL 17 to 18. The /var/lib/postgres/data</code> directory was originally setup with 17. On Arch Linux the way that pacman works with upgrades is that it upgrades to the latest. So when you upgrade postgresql it will have only the latest version installed. But in order to facilitate upgrade with postgresql you have to have the previous version installed already. To facilitate this Arch User Package's available through yay</code> provide a package for the previous version to be installed beside the current. The following facilitates installing these packages. yay -Sy postgresql-old-upgrade yay -Sy postgis-old-upgrade # if you were using postgis with postgresql in the previous version</code></pre> Once these are installed we need to move the old data dir to a backup. This can be done as follows. sudo -iu postgres sudo -u postgres mv data data.old17 exit</code></pre> Then we need to create new initialized data directory for the new version of the database. Note: The --no-data-checksums</code> option is something I added because the 17 data was done without data checksums and the pg_upgrade</code> process doesn't allow upgrading from a non-checksum on to checksum one. sudo -iu postgres mkdir data initdb --no-data-checksums -D /var/lib/postgres/data exit</code></pre> Once you have the new data directory initialized then you can run the upgrade with the following. sudo -iu postgres pg_upgrade -b /opt/pgsql-17/bin -B /usr/bin -d data.backup -D data exit</code></pre> Then you can start the service up again. sudo systemctl start postgresql</code></pre> Some statistics are not transferred by pg_upgrade. Once we have started the new server we should be able to do the following to handle this. sudo -iu postgres /usr/bin/vacuumdb --all --analyze-in-stages --missing-stats-only /usr/bin/vacuumdb --all --analyze-only exit</code></pre>Pull Request Alternative Design Doc 2025-02-07T00:00:00+00:00 The following is a Design Doc for an idealized alternative to what is currently the pull request work flow. The Problem</h2> The feature branch/pull request process is fundamentally broken a bunch of different ways. This can be improved significantly with a Patch Stack</a> based workflow and tooling. But this is still far from ideal as when you have dependent patches you can't open a review for the higher level patch until the dependent patch is integrated into mainline. A stacked branch approach can elevate this but introduces other problems. Like, having to manage branch names up front as well as manage the DAG (directed acyclic graph) of the relationships between the branches. Also, it requires a completely different service that companies have to pay for to extend the peer review process, see Graphite</a>. Idealized Goals</h2> open patch review requests as soon as completed, even if they are patches that depend on another patch that has not been integrated into mainline yet.</li> open patch review requests for a patch series communicating the dependence and order of dependence.</li> don't require an additional service</li> promote better reviews by actually having people use their own tools to review</li> support reviewing commits after they are integrated into mainline as well as before they are integrated into mainline</li> </ul> The Thinking</h2> The initial high level thought is, that if we were to use git format-patch</code> or similar to facilitate the review process. Then the review would be free floating, or independent, even if it actually depended on another patch. This can could be seen as a drawback. But in reality it is a benefit as it effectively lifts the requirement of that dependency from the point of requesting the review and moves it to the application of the patches. This would allow the developer who wrote the patches to effectively request review for each of them independently even if they had dependence of another patch that has yet to be integrated into mainline. This might not eliminate the concept of a patch series all together. But it gives you flexibility and optionality in terms of workflow. pr.pico.sh</a> is the closest thing to this that I have come across. Though, it is a separate service that you would need to host. Git Commit Creation 2024-10-16T00:00:00+00:00 When people are first learning Git they seem to learn something along the lines of the following. You can create a commit by doing the following. staging a change with git add <path/to/file></code></li> create the commit with git commit</code></li> </ol> </blockquote> Or maybe something like. You can stage and commit in one shot with git commit -a</code>. </blockquote> It seems that for the majority of people, once they learn the above about creating commits, they just move on and never look back. In my opinion this moving on and never looking back has massively delayed individual growth as well as hindered the development of maintainable systems. So with this article I am attempting to provide the ultimate guide to Git commit creation. In the hopes that it will help newer developers understand everything that they should be taking into consideration when creating a commit. And who knows, maybe we will be lucky and a couple of those devs that haven't looked back, will slow down for a minute and think about creating Git commits again. The Pressures</h2> To really start to understand how we should be creating commits, and the reasons why. We need to explore the pressures imparted on Git commit creation from Git itself, the needs of our team members, both right now and in the future, as well as the needs of our selves, both right now and in the future. Anatomy of a Commit</h3> To start we really need to understand what commits are in Git. Technically from a Git perspective a commit is a snapshot of a project's contents, files & folders, at a moment in time combined with a message. Snapshots & Diffs</h4> In the Git repository these snapshots are related to one another through the concept of ancestry. Meaning that generally each commit has one or more parent commits. These parent commits are the ancestors of the commit. Side Note: There is technically a special case of commit that doesn't have any parents. Commonly you will see this with the first commit in a repository. However, you can actually create multiple of these within a repository if you like. If you are curious this is generally referred to as an "orphan" branch and can be done with git checkout --orphan</code>. See the git-checkout man page for more details if you are interested. </blockquote> So we have snapshots with messages, and they have ancestors. What is the big deal? Well, the natural thing that falls out of a snapshot having an ancestor is that there is a difference (a.k.a. diff) that exists between two snapshots. In fact this is such a prominent mental model that when you do a git show <sha></code> it shows you the difference between the ancestor commit's snapshot and the snapshot of the commit identified by the given <sha></code>. A lot of people choose to simply think about commits as diffs even though they are actually storing full snapshots of the project in the Git repository. Commit Message</h4> The message portion of the commit is an extremely important and valuable piece of information, that if used properly, facilitates code reviews, as well as short-term and long-term maintenance and understanding of a code base. Git actually has an official format for Git commit messages, which is as follows. Short summary of change (<= 50 chars in len) One or more paragraphs of text. (hard wrapped at 72 characters)</code></pre> If you are working with any editor worth its salt. It will have support specifically for the Git commit message format and help you abide by the constraints of this format. Abiding by this format is important as it will make sure that our commit messages work within Git as a tool and it's expectations. Commit Characteristics</h3> In addition to understanding the anatomy of a commit, their associated requirements and implications, it is also important to understand the expected characteristics of a commit within Git. When you are making a commit. Two characteristics that are important to make sure you take into consideration are that the commit is buildable and testable. Buildable simply means that the snapshot that will be captured by the commit, is able to build without failure, using the build system of the project. This is applicable not only to compiled languages but also can be conceptually extended to include other tooling like linting and formatting. Testable simply means that the snapshot that will be captured by the commit, is able to successfully run the automated tests without failure. These are crucial for a number of reasons. For one, Git provides tools that depend on these characteristics being met. For example git bisect</code>, an extremely valuable tool that aids in automating finding a commit that introduced a particular issue. If your commits aren't buildable and testable git bisect</code> doesn't work in an automated fashion. When your commits are not testable your team will quickly lose faith in the value that the test suite is providing, which will generally result in people's testing practices waning, likely due to the Broken Windows Theory. Beyond that if your commits are not buildable your teammates will quickly become irritated with you because your commits will be impeding them from making progress on the things they are trying to push forward. Scope & Intent</h3> At this point we know that that commits have a snapshot of the project and an associated message that should abide by the official format, and that the commit should be Buildable and Testable. However, we don't have any real guidance in terms of what should be included in the snapshot and the commit message. This is where scope and intent come into play. Scope and intent are the key to one of the most important software engineering practices of today, understanding and properly defining commits. When we are making changes to software it is crucial to think about the intent behind the change. This can also be thought of as simply answering the question, "Why was this change made?" However, thinking about the intent alone is not enough. We also need to think about the scope in terms of the various irreducible (a.k.a. atomic) components of the proposed change. This is due to the fact that the intents of a change are the combination of the individual intent of each irreducible component. This sounds a bit theoretical at this point. But it is actually something that is quite concrete and is very easy to see when simply looking at a pull request's changes. All we have to do is ask ourselves the question, "Why was this change made?", and then as we are looking at the code, validate whether that specific code change aligns with that intent. I recently reviewed a pull request that was created to tackle a particular bug. However, when looking at the pull request there were a lot more changes than I expected to see. I wasn't sure why those changes were there, but they didn't seem to be required to fix the bug. At this point there is no way anyone should sign off on this pull request as we really have no idea what it is doing, let alone, why these changes were made. So I talked to the developer that made the pull request and through conversation we determined that the changes that weren't specifically fixing the bug were made for a handful of different reasons, to eliminate duplication, to fix linting errors/warnings, and to clean up a toggle. It looks like we have a bit of a miss alignment here in terms of intent because we have code that exists with intents other than fix the bug. More accurately, we have a bit of scope creep. If we look at the pull request visually in terms of intents and scope it would currently look something like the following. In the above we can visually see that in this case the current scope of the commit involves the following intents. Commit A literal bug fix</li> clean up toggle</li> fix lints</li> correct duplication</li> </ul> </li> </ul> But, it is a single commit and that commit only has one scope. You could theoretically argue that you can include in the commit message details describing that the commit addresses all these intents. But that would not address the issue that when looking at the code you wouldn't be able to distinguish if a line of code was changed to fix a linting rule or changed to address the duplication, etc. So a much better approach is to create separate commits, one for each of the intents. This binding between an intent and a commit, which has a singular scope, is a crucial underpinning to building maintainable software systems efficiently. The above visually represents each scope as a different commit paired with each respective intent. So we would effectively have the following commits. Commit A: literal bug fix</li> Commit B: clean up toggle</li> Commit C: fix lints</li> Commit D: correct duplication</li> </ul> This fundamentally makes the peer review process go much quicker as the commits are focused on a singular narrowed intent, and provide the critical context in the commit message. Intent & Architecture</h4> As the old adage says, "There's more than one way to skin a cat." That statement is true in the context of defining intents as well. However, that doesn't mean that all the ways of defining intents are equal. For example. We could take and break out our commits for a project as follows. initial set up of listing action v2</li> added api calls, and computed variables to allow for checking which CTA to display</li> finished functionality for all of the child components</li> </ul> If we look at the summaries we can maybe draw the following diagram based on understanding. If we just look at the summaries and diagram above we can gather the following. That there is some initial setup. Though we are not sure what that is exactly without looking deeper.</li> That some API calls and computed variables were added to facilitate checking which CTA to display. But we don't have any information about what component(s) are responsible for/involved in this.</li> We know some sort of work has been done to finish the functionality for all the child components. But we don't know what that is without looking deeper.</li> And we know that there is some concept of listing action v2 but don't really understand how it relates to anything else.</li> </ol> The above is technically broken out into intentions. The first intention was, "setup listing action v2", the second was "add api calls & computed variables to allow for checking which CTA to display", and the third was "finishing up some functionality". It is worth noting that the second intention has a smell as it has the keyword, and, in the description. Hinting there are probably multiple intents within the scope of that commit. Despite multiple intents in that scope, this is definitely a step in the right direction as the commits are broken out into intentions. But we can do much better with a slight shift in our thinking about intents. If we add the constraint that our intents should generally be related to application architecture concepts. Things like modules, classes, methods, functions, services, etc. We would have likely defined the intents and commits as follows. add can apply, reason props ProfessionalViewListing</code></li> mod ProfessionalListing</code> API client to map props</li> add listing actions components (e.g. AcceptShift</code>)</li> add ListingActionV2</code> to show proper action component</li> register the listing_action_v2_fe</code> toggle</li> mod the Listing</code> view to show ActionV2</code> or ActionV1</code></li> </ul> This is fantastic as now we have lots of information just from the summaries of these commits. In fact, it gives us enough information to draw the following diagram of the architecture involved. So from the above diagram we can see that the commit summaries containing intent around different architectural concepts, as an approach, has provided us with a lot more value in terms of understanding changes as well as the code base itself. In particular, we know that there is some sort of collection of Listing Action components and that the ListingActionV2</code> component is responsible for choosing which one to render based on some criteria. We also know that the Listing View</code> exists and that it conditionally chooses to render either the ListingActionV1</code> component or the ListingActionV2</code> component based on the registered listing_action_v2_fe</code> toggle most likely. We also know that there is a ProfessionalViewListing</code> that received the can_apply</code> and reason</code> properties. As well as that there was a ProfessionalListingApiClient</code> which was modified to map properties, which are most likely the properties that were just added in the previous commit. We were almost able to build a complete architectural diagram expressing the relationships between the various components involved just from the commit summaries. The one thing we are missing is just how the API client relates to the rest of the architectural concepts. It is amazing how valuable this is and how much it facilitates rapidly gaining context around an area of effort. So, I hope this makes it clear that even though we can define intents in a bunch of different ways. We generally should define them based on architectural concepts due to the massive gains that we get. Fallback on Scope & Intent</h4> Now, you may have noticed that I said aligning your intent is generally the right thing to do. That is because there are scenarios where it may not be fitting. An example, where you would likely want to fall back to simply using Scope & Intent to define your commit, is when you are switching from not having automated formatting in a project to having automated formatting. In this case you could try and argue that you could format sections of the code based on architectural concepts. Though, generally this isn't practical due to the fact that the tooling doesn't make it easy to apply auto formatting only to an architectural concept within the code, and secondly because most of the time people don't have a solid well organized architecture to help with this. So instead you create a commit that formats all the files in the code base. It could be thousands of files, but you get it done in one fell swoop. And you make the summary of the commit clearly communicate the scope & intent of formatting all the files. A similar scenario would be enabling linting across a project that didn't have linting before. In my experience scenarios that are incompatible with architectural alignment are pretty rare. So we should always make sure we are putting effort in to strive for architectural alignment and only fall back to solely scope and intent after exhausting all the ways to facilitate architectural alignment. More than a summary</h3> At this point we have great guidance in terms using scope & intent to define our commits and their commit summaries. And we have learned how valuable a commit's summary can be when the commit is defined with a singular intent around architectural components. However, the commit summaries are limited to 50 characters, or less. Which makes it difficult to fully communicate the intention of a change. Let alone provide any additional information about the approach that was taken and why. Luckily, the Git commit message format supports providing any number of paragraphs of detail as long as they have blank lines separating them, and they are hard wrapped to 72 characters. This allows us to define our own Git commit format based on the official Git commit format plus our learnings from above. The following is a Git commit message format I have refined over the past 24+ years of development experience. Short summary of change (<= 50 chars in len) One or more paragraphs explaining the INTENTION & REASON for the change. (hard wrapped at 72 characters) One or more paragraphs explaining the APPROACH taken to attempt to achieve the above described INTENTION or a portion of the INTENTION. (hard wrapped at 72 characters) Associated Ticket Identifiers</code></pre> The short summary should describe the change. I like to think of this as the what of the commit. It is valuable to give a high level understanding of what change the commit represents. The intention & reason paragraph on the other hand is by far the most valuable part of the commit message as it is something that is impossible to get from simply reading the code. I like to think of this as the why of the commit. Following that is the paragraph covering the approach. This is something we can kind of get from reading the code, but not entirely. So we include this as additional context in the commit message. You can think of this as the how of the commit. Lastly, we provide any associated ticket references. This facilitates ticketing systems tracking commits and their association to tickets in the ticketing systems. This is an extremely beneficial thing to include as it facilitates going back and finding even more context about a change. So for every commit we provide the what, why, how, and associated tickets so that peers reviewing the changes have all the associated context and so that in the future if we, or another dev, need to understand the intent and context of a change we can easily get it from the commit message. Proof of Work</h3> When you create your commits taking the above into consideration another natural benefit appears. It is something that prominent development teams, like the Linux Kernel and Git development teams, talk about in respect to their Git commit requirements. It is this concept of Proof of Work. The idea is that when you submit a series of commits in a particular order, the commits and the order should communicate the steps you are taking to accomplish the higher level goal & intent of the pull request or patch series. Think of the commits being the equivalent of writing out the steps you performed when solving a math problem as a kid. In fact, it carries the same benefits. It helps you and others see how you got to the resulting answer. And if that answer isn't 100% correct, it helps you or others see the various steps you took and be able to investigate or validate each one of them. Additionally, if we are following the practice above of binding scope and intent to architecture, then it also provides an understanding of how the greater intent was accomplished in the terms of the application architecture. This is extremely valuable from both a peer review standpoint, and from the standpoint of trying to understand changes at a later point in time. Tactics</h2> We have covered some of the details about what commits are, what their contents and message should contain, and the arguments backing them. However, we haven't really started covering how you actually go about creating commits like this. So let's shift more into the mental hurdles and the tactical skills needed to actually create better commits. Commit Permanence</h3> One of the biggest hurdles people seem to have when shifting from making horrible commits to making great commits, is breaking this misconception that commits are permanent the moment you create them. I don't know where this misconception comes from. But in my experience it is quite prevalent. But it is clear that it doesn't come from Git proper. You can see this fairly quickly just by looking at some of the operations that Git facilitates you performing on commits. git commit --amend</code></li> git commit --fixup reword</code></li> git commit --fixup amend</code></li> git commit --squash</code></li> git commit --reset-author</code></li> git commit --author</code></li> </ul> The above is all just in the git commit</code> command. There is even more once we look into git rebase</code> and other commands. From looking at the above set of commands alone, it is clear from a Git perspective, that commits are made to be modified & mutated at least up until some point in time. It is also clear that, that point in time is not at the moment of creation of the commit. So what is that moment of time when a commit should become permanent? I would argue that it is the moment that a commit is integrated into the mainline branch. Prior to that moment the commits should be free to be refined and reworked. Isn't Rewriting History Bad</h4> If we take that stance though then we have to be ok rewriting history, even once it has been shared. And isn't rewriting shared history a big no no? Yes and no. Let me explain. There are cases where it is giant pain in the butt and can be extremely disruptive to other developers if you rewrite shared history. One such scenario is if you were to rewrite the history in the published mainline of a repository. In general, it is bad form to rewrite the history of mainline of a repository. Although, sometimes it is necessary and worth the disruption, e.g. to strip accidentally shared credentials, etc. So just like everything else in engineering. There are pros and cons. We just need to make sure we understand the pros and cons and choose the path that fits our needs the best. Rewrite Pull Request History</h5> If we look at the scenario of rewriting history in a shared pull-request branch. It enables us to have commits that can communicate a ton of context and value in terms of architecture. Additionally, it facilitates reverting specific targeted changes if necessary. As well as answering questions about the intent of changes over time in relation to architectural concepts. And bisecting to identify a specific change that introduced an issue and what the intention of that change was. Just to name a few. At the same time it has the drawback that it can break direct code collaboration if there isn't communication between the parties. Generally though, there isn't much direct code collaboration on pull requests as the author is usually the one making the changes. And if the reviewer wants to make a change and push it up to the pull request branch. All they really need to do is let the pull request author know that they have pushed up a change, so they can manage integrating it into the final commit. Squash & Merge</h5> Another approach is to take a hard-lined stance about not rewriting any shared history, including pull requests. This means that the only way we can bring in changes from mainline is to back merge mainline into our branch. It also means that we can't refine our commits once they are pushed up to a pull request. In turn requiring us to add additional commits to the pull request that are not logically structured and are not representative of the application architecture or the intents at all. This in turn results in a bunch of commits, that are meaningless to the Git history existing within the pull request branch. And given that these commits are generally not buildable & not testable they result in breaking git bisect</code> and other functionality. To help mitigate this people will do a "Squash & Merge" in which all the commits within the pull request branch are squashed together into a singular commit. This helps resolve the buildable & testable characteristics in the majority of cases. However, we have completely lost the ability to revert targeted changes and the ability to even understand the intention of the changes from an architectural perspective. The only thing we have gained from this is "smoother direct code collaboration". Meaning that in the more rare scenarios when you are reviewing a pull request, and you want to push a change up to it. You don't need to let the author know that you are pushing a change up to their branch. Although you generally do anyway. The Choice</h5> The choice is pretty clear to me as the overhead of pinging the author via messaging is trivial, as a cost for rewriting pull-request history. And it unlocks a wealth of benefits. Enabling our commit's moment of permanence to be when the commit is integrated into the upstream mainline. If you want to get a deeper understanding of this in relation to the Git tree. Check out my article, When to rewrite Git history</a>. But How do we do this?</h3> You might be thinking. How do I actually accomplish this? Learn Git</h4> Well the extremely short answer is simply, learn more about Git. It really boils down to learning a few commands, git add -p</code>, git commit --amend</code>, and git rebase -i</code>. It is worth noting that it is crucial that you have a solid understanding of Git commits and the Git commit tree in general when truly understanding these commands. The following is an outline of the related operations you would perform with each of those commands and links to respective documentation or guides. Note: I included links to guides for Git Patch Stack</a>, a Git extension that facilitates a patch stack workflow with Git, because it documents the main commit mutation operations with gps rebase</code> which is really just a convenience mechanism for git rebase -i</code>. creating multiple commits from uncommitted changes Add hunks instead of entire file git add -p <patch/to/file></code> git add -p manpage</a></li> </ul> </li> Use a Git GUI to stage individual lines or selections My article on git add -p won't split</a></li> </ul> </li> </ul> </li> amending a commit git commit --amend</code> git commit --amend manpage</a></li> </ul> </li> git rebase -i</code> ProGit Book - 3.6 Branching - Rebasing</a> git rebase -i manpage</a> edit a commit Git Patch Stack: Docs - Edit a patch</a></li> reword a commit message git rebase -i</code> and mark for reword</code></li> </ul> </li> reorder commits Git Patch Stack: Docs - Reorder patches</a></li> drop a commit Git Patch Stack: Docs - Drop a patch</a></li> add a commit in the middle Git Patch Stack: Docs - Add a patch in the middle</a></li> fix a commit up into another commit Git Patch Stack: Docs - Combine multiple patches</a></li> squash commits together Git Patch Stack: Docs - Combine multiple patches</a></li> split a commit up into multiple commits git rebase manpage splitting commits</a></li> My article on Git - Splitting Commits</a></li> Git Patch Stack: Docs - Split a patch up</a></li> </ul> </li> </ul> </li> </ul> WIP Commits</h4> In addition to learning Git. There is another concept that is extremely valuable now that you are living in this world of commits that you evolve over time. And that is the WIP (Work in Progress) commit. This is simply a convention of prefixing the commit summary with WIP:</code> to let ourselves know that the commit still needs further refinement before we submit it in a pull request or integrate it into mainline. Once we have refined it to the point that we are happy we simply update the commit summary, removing the WIP:</code> prefix. In turn finalizing our commit conceptually. Commit Message Template</h4> Beyond that. There are things that are just hard to remember. The various parts of the commit message can be one of those things. Luckily Git helps us out by providing support for commit message templates via the commit.template</code> configuration setting. If you want to set this up you can do so by running the following command. git config --global commit.template ~/.gitmessage.txt</code></pre> The above configures Git to look for the commit message at ~/.gitmessage.txt</code>. This template file is simply a text file that is loaded up into the editor when you go to create a commit. The contents of my ~/.gitmessage.txt</code> are as follows. # Short summary of change (the what) ( <= 50 chars ) # INTENTION of the change & REASON for change # (Hard wrapped at 72 chars) # APPROACH taken & WHY in relation to attempting to achieve INTENTION # (Hard wrapped at 72 characters) # Associated Ticket Identifers</code></pre> The layout I provided is based on making it efficient to enter. Starting out when it loads it in the editor it drops you into the top left corner ready to start writing your 50 character or less summary. It is important to realize that any line in the Git commit message starting with a #</code> is considered a comment and will not be included in the final commit message. This means that I don't actually need to replace any of these comments. Instead, I just add the respective content below each comment. The one exception to this is the short summary which is added above the comment instead of below. It is also worth mentioning that you will want to keep a blank line between each of these sections. There are many more things you can include in your commit messages and the template to help you. Some people add explicit change log entries to their commit messages that are targeted at the consumers of the product rather than the developers. Others manage sign-off from peer developers through commit message standards they have defined, etc. In my opinion, all these things can be valuable, and should be considered, but the above template really outlines the core that should always be present. Buildable & Testable Enforcement</h4> The buildable and testable characteristics are another thing that are easy to let slip by if we are just manually verifying them. To help us hold to these requirements some people set up a Git pre-commit hook to make sure that each commit is buildable & testable prior to commit creation. In my experience this isn't the ideal enforcement mechanism because it assumes that commits are final and that you aren't going to iterate on them or evolve them. Which we have seen above is pretty damn important. A more appropriate hook would be a hook that is triggered when you want to either request that someone review a commit or integrate a commit. In turn helping make sure that any commit you are going to share either via integration into mainline or via pull request/patch is buildable and testable. Git doesn't natively have such a hook. But if you use Git Patch Stack</a> it easily supports this via it's isolate_post_checkout hook</a>. You might be thinking. I don't need to worry about this. We have a CI (Continuous Integration) service that makes sure my pull requests are both buildable and testable. But in general this isn't a valid enforcement mechanism as most teams don't strictly only create pull requests consisting of a single commit. So if we make sure that our commits are buildable and testable it in turn guarantees that our pull requests are buildable and testable. However, if we only make sure that our pull request is buildable and testable it is quite possible, if not likely, that one of the commits in the pull request in not buildable or testable. Outside-In</h4> Outside-In is an engineering practice in which you drive changes from the outermost thing first and work your way inward. Generally, the outermost thing is the UI layer. The massive benefit of this is that you always end up building only what you need to build as it is driven by the need of the layer above. This is in opposition to inside-out or middle-out development in which you try and guess the needs of the outer layers and build the inner/middle layers first. Generally this results in your guesses being not quite right and having to do rework or even worse including additional complexity and functionality that isn't needed by the layers above. Outside-In is extremely useful in helping you think about the architecture concepts. As you are effectively working from the outermost architectural concepts, to the innermost architectural concepts, and then working your way back up from the inside-out, connecting each of the layers. Using WIP</code> commits can aid this process as your outer layers can start as WIP</code> commits until you reach that innermost layer which ends up being a complete logical chunk. Then as you work your way back up the layers connecting them to the layer below you are effectively finalizing them and getting rid of the WIP</code>. To get a better understanding of Outside-In with an example you can check out my article, How we should be using Git</a>. Conclusion</h2> At this point you should hopefully have enough knowledge and context to be able to start creating meaningful, logically chunked, buildable and testable commits that are bound to scope, intent and architecture. Resulting in a valuable Git repository history that will aid you through the short-term and long-term maintenance and development of the project. When to rewrite Git history? 2024-06-13T00:00:00+00:00 Ok, so let's talk about Git and this "rule" that people often get attached to, "Don't rewrite history once it is shared!" There is definitely some wisdom in this "rule". However, following this "rule" religiously also has some side effects. And it is worth understanding those in more depth, as well as understanding any alternatives and what their pros & cons might be. When you follow this rule we have to ask, how do the branches we are working on get updated with changes other developers have already integrated into main</code>? Well, there are a couple of approaches we can use depending on our situation. To explore these, let's assume we start with a Git tree that looks as follows. * 9cabcd2 - (HEAD -> main, origin/main) commit C * 6327c81 - commit B * b68a192 - commit A</code></pre> Now let's assume we have created some changes in a branch locally that we have not pushed up to the remote yet. And that someone else has integrated another change into main</code>. * a0aca6f - (main, origin/main) commit G | * fe06e90 - (HEAD -> feature/x) commit F | * 81312b5 - commit E | * f09acea - commit D |/ * 9cabcd2 - commit C * 6327c81 - commit B * b68a192 - commit A</code></pre> One option we have to include the new change in main</code> as part of our feature/x</code> branch is to use git rebase</code>. This can effectively take the commits from our branch and replay them on top of the commit that was added to main</code>. Our tree would look as follows after doing this. * ae46190 - (HEAD -> feature/x) commit F' * 512b2ba - commit E' * d09efea - commit D' * a0aca6f - (main, origin/main) commit G * 9cabcd2 - commit C * 6327c81 - commit B * b68a192 - commit A</code></pre> This works great! It keeps our tree clean, simple, linear, and easy to understand. However, commit D</code>, commit E</code>, and commit F</code> all have different SHAs now, and are effectively commit D'</code>, commit E'</code>, and commit F'</code>. This is totally fine right now because we hadn't shared commit D</code>, commit E</code> and commit F</code> with anyone yet. However, if we had shared commits D</code>, E</code>, and F</code> already by pushing our branch up to the remote. We would have technically shared that history. And our Git tree would look as follows. * a0aca6f - (main, origin/main) commit G | * fe06e90 - (HEAD -> feature/x, origin/feature/x) commit F | * 81312b5 - commit E | * f09acea - commit D |/ * 9cabcd2 - commit C * 6327c81 - commit B * b68a192 - commit A</code></pre> Therefore, when following this rule we can no longer use git rebase</code> to base our branch on top of the latest main</code> because it would accomplish that by rewriting the commits in the branch. Rewriting the commits could create complications for other developers who happen to be working on the previous state of the commits as the new commits have now diverged. Instead, we need to bring the new commits in main</code> into our feature/x</code> branch in a way that doesn't rewrite history. git merge</code> is exactly the command we are looking for. As it creates a new merge commit that has multiple parent commits. If we ran git merge main</code> it would look something like the following. * f1cca4d - N - (HEAD -> feature/x) commit H |\ | * fe06e90 - N - (origin/feature/x) commit F | * 81312b5 - N - commit E | * f09acea - N - commit D * a0aca6f - N - (main, origin/main) commit G |/ * 9cabcd2 - G - commit C * 6327c81 - N - commit B * b68a192 - N - commit A</code></pre> In the above Git tree we can see that feature/x</code> contains the same commits with the same SHAs as before, but also contains a new commit that merges main</code> into feature/x</code>. This is great as it makes it so that you can easily collaborate with one or more developers on the feature/x</code> branch. However, it also made the Git tree more complex and complicated to understand. It might not be too bad with one branch. But we haven't gotten to the step of merging our branch back into main</code> at some point in the future. Or, the reality that often you need to merge main into your feature branch multiple times as new changes are introduced. Or, the reality of dealing with multiple branches doing the same thing at the same time. Generally, you will end up with a very complicated tree that looks something like the following. Having these forward & backward merges happening also ends up eventually confusing Git when it comes to its ability to find merge bases correctly, in turn limiting the functionality of Git. What is the Alternative?</h2> The alternative is to simply use git rebase</code> on your feature branches. However, we know that this can create problems for those collaborating on a feature branch. So we simply have an understanding of how we are going to treat feature branches by default. For example, we can say that by default feature branches are owned and controlled by their author. Anyone pulling down that branch and testing it out or reviewing it should understand that the history is owned by the author, and they will likely change it. Generally, the reviewing developer uses something like git reset --hard origin/feature/x</code> to update their local state to match that of the remote state. Now, if you communicate out of band with the developer that owns the feature branch and let them know that you are going to push up a commit, they can integrate it into their feature branch in their local git repo and still use rebase. It just requires some out of band coordination. But this is actually a relatively rare situation for this to occur and therefore isn't a big deal. It is also worth noting that taking this approach allows you to use the "Rebase & Merge" strategy in GitHub which will result in a linear Git history. Making it much easier to understand and manage over time. Conclusion</h2> I have used both these strategies and many others throughout my career. But the ease of understanding & management of a linear Git tree is far more valuable than simply not needing to worry about a tiny bit of out of band communication in the rare scenario that you need to directly contribute to someone else feature branch. The key with this alternative is that you don't rewrite history for main</code> as it is a central point of collaboration where you don't want to have deal with the overhead of the out of bound communication and coordination. But due to the rareness of the need to directly contribute to other peoples feature branches the overhead of the out-of-band communication is more than an acceptable trade. Reasonable Code Organization 2024-03-27T00:00:00+00:00 The following are some excerpts related to code organization from a code review I did recently. I started reviewing this PR and wasn't quite sure exactly what the separation of concerns were with the boundaries you had drawn. So I drew a diagram to try and help myself understand better. It ended up looking as follows. From looking at the code in far more depth I started to get a better understanding but still wasn't sure about things. Largely, what component was responsible exactly for what concerns. I figured out through reading a bunch of the code that EventStreamQueryService</code> was for querying Athena for Events. Which roughly lines up with the diagram above. Part of what threw me on that was the naming having Stream in there. The following questions were flowing through my head. what does Stream</code> have to do with querying?</li> is it integrated with a stream somehow?</li> why is Athena a peer?</li> is something else using it?</li> </ul> Beyond that even when I looked at the code of the Athena service it has knowledge of Events</code> which isn't an Athena thing at all. So, I figured I would throw a quick diagram together to show how I would think about this problem architecturally and how I would draw the lines. That looks as follows. I tried to explain the responsibilities of each of the concepts in the text next to it. The arrows represent their relationship in terms of dependence (what things depend on what things). This generally helps in terms of making decisions about encapsulation of dependencies (a.k.a. are you going to hide a dependency away or are you going to expose it intentionally for other things to use it). This naturally leads to the question. "How do you organize this code so that it is far more consumable and easy to understand coming into the project?" That is where modules come into play as they facilitate encapsulation of dependencies as well help to formalize interfaces. It is worth noting that I am not talking about Nest.JS modules, though they can also facilitate encapsulation. I am simply talking about standard Typescript modules. If we take the example above you could create a directory structure as such. Having a directory structure like this is extremely valuable because of the encapsulation. It tells me that as a consumer and developer of the Event Stream Module</code>, that the Event</code> and EventStreamReplayService</code> are the public interface and entry point for the module. I know this very quickly just from looking at the directory structure. Everything else is encapsulated and hidden away with in their respective modules. This is also extremely useful to know as a developer because I can understand for example that EventStreamService</code> depends on KinesisService</code> but that nothing else should because it is encapsulated within the EventStreamService</code> module. The same goes for EventQueryService</code> module. It encapsulates the Athena Service</code> telling me as a developer nothing else should be using that Athena Service</code>. Instead, they should be going through the EventQueryService</code> or if some functionality doesn't make sense being added to EventQueryService</code> then the AthenaService</code> should be elevated up to a higher level shared dependency. All of this, aids massively in communicating the boundaries and their actual association to the architecture itself. Something like what we originally had doesn't tell us much. It tells me that a StreamEvent</code> exists that a EventStreamQueryService</code> exists, that a EventStreamService</code> exists, and that a AthenaService</code> exists, but it doesn't tell me much of anything about their relationships or what their responsibilities are intended to be. Not having clear boundaries allows you to easily fall into traps that make your code far less maintainable and reusable. For example if we look at the AthenaService.fetchQuery()</code> method. async fetchQuery( queryExecutionId: string, nextToken?: string, ): Promise<Result<EventStreamQuery, AthenaServiceError>> { try { const data = await this.client.send( new GetQueryResultsCommand({ QueryExecutionId: queryExecutionId, NextToken: nextToken, }), ) const events: EventStreamEvent[] = [] data.ResultSet?.Rows?.forEach((row) => { events.push({ eventSource: row.Data[0].VarCharValue, projectId: row.Data[1].VarCharValue, occurredAt: new Date(row.Data[2].VarCharValue), payload: JSON.parse(row.Data[3].VarCharValue), }) }) return Ok({ queryExecutionId: queryExecutionId, state: EventStreamQueryState.SUCCEEDED, nextToken: data.NextToken, events: events, }) } catch (error) { return Err(this.mapError(error)) } }</code></pre> It is building a EventStreamQuery</code> object as well as building an array of EventStreamEvents</code>. This is a mixing of two concerns here. One is the work necessary to deal with Athena client and the other is work that is structuring data to be Event</code> data. Let's say we decided to use Athena</code> for something else in the future in this project, or who knows maybe in another project. Because these two concerns are mixed here it would require much more work to rebuild all the context around this problem space and safely do the extraction of just the Athena</code> specific piece that could be reused. Keeping those boundaries clean now doesn't cost very much at all as the context is already present. If you aren't clear on TypeScript Modules</a> it is a standard language feature that is worth understanding. The pattern of having a directory represent your module with a index.ts</code> file in it can be very useful. To complete the exercise with the same type of diagram I started with, but in this proposed new world order. It would look like this. This is of course possible to do with Nest.JS modules. However, it doesn't really give you any advantage and adds more boilerplate code and complexity than is necessary. I hope this has been useful and am always open to discuss further. My Keyboard 2024-01-18T00:00:00+00:00 TLDR</h2> The following is a deep dive into my current keyboard which is outlined below. 4x5 Dactyl Manuform (with a few small tweaks)</li> Printed on my 3D printer</li> Custom-built</li> Hand Wired</li> Cherry MX Blue switches (Cherry MX Silver in my secondary keyboard)</li> Firmware: QMK</a></li> My fork of QMK firmware (repo: git://git.drewdeponte.com/qmk_firmware.git</code>)</li> Dvorak</li> Pretty Layout for Printing</a></li> </ul> Why a Dactyl Manuform?</h2> Well, the short answer is because I have historically had wrist pain from typing. But not with the 4x5 Dactyl Manuform. I believe at this point with all my experimentation with a ton of other custom keyboards I have built that the key to my success with this build is due to two things. First, I determined that my wrist pain was caused largely by repetitive stretching of my fingers to reach keys at the outer boundaries of keyboards. The only way I could figure to reduce that was to reduce the number of keys I have on the keyboard. Hence, the 4x5 size. However, this meant that I would have to have layers and a comfortable thumb cluster I could use all the keys from. The Dactyl Manuform 4x5 meets all these requirements and still maintains all my other ergonomic requirements. Also thanks to QMK it is customizable enough to support a layered layout in a way that I can still be efficient at programming. A Problem, No More</h2> I love my Dactyl Manuform keyboard. It is by far the best keyboard I have every used and my wrists haven't felt any pain since I started using it. However, it has had one problem. It is too light-weight. Because of this even when I have rubber on the bottom of it, it still slides around on my desk. The other day I decided it was time to address this annoyance. So I took them apart and first tried to see if I could figure out a good mechanism for adding weight to them. Didn't have much luck with that. And then I thought my other issue is having my trackpad positioned appropriately relative to the keyboard. This got me thinking if I could create some sort of thin thing that would hold the keyboards in the appropriate positions to be ergonomic, as well as hold my trackpad. So I laid out the components on my desk in the correct positions and then took a rough measurement to figure out how large something would need to be. I then took those measurements to find some scrap Masonite board in my wood shop and cut it down to that size. Then I laid the components on it the correct positions and then used some CA glue to glue down tiny blocks to hold the components in the correct positions & orientations. This actually worked extremely well. It kept the keyboards and trackpad in the correct positions. It also increased the surface area in contact with the rubber as well as the weight. Causing it to prevent the devices from sliding around on my desk. However, the next problem that I had was that it was ugly and that it bothered my wrist when I would relax my hand on the desk. Beyond that it was just ugly. So I free hand drew a more organic looking outline around the components, visible in the photo above, and then cut it out with a band saw. This made it not only look nicer. But also eliminated the discomfort that I had when I would wrest my hinds on the desk every so often. In fact, you can see what it looks like in use in the following photo. It is still a bit ugly and could use some love. So my plan is to cover up the blocks and the visible wood by sculpting clay to make in organic looking alien blob that is absorbing my keyboards and touchpad. And then I will paint it. Hopefully, at that point it will look much nicer. The Build</h2> The following are a bunch of photos I took throughout the build. I won't go into all the details about them. But figured I would share. Notes</h2> The following are really more notes for me about things. But it might be useful to you if you are trying to get a Dactyl Manuform setup with QMK Firmware. References</h3> abstracthat/dactyl-manuform</a></li> </ul> Parts</h3> Tapered Heat-Set Inserts for Plastic</a> (Brass, M3 x 0.50 mm Thread Size, 3.800 mm Installed Length) - use for attaching a base to the bottom of the housing</li> M3 wafer-head screws, 5mm</a> - used to attach the base to the inserts in the housing</li> 1N4148 diodes</a></li> #30 wire</a></li> 2 Arduino Pro Micros</a></li> Veroboard stripboard</a></li> 4pin Telephone Cord</a></li> Female RJ-9 connectors</a> - 4pin RJ-9 Female Connectors</a></li> </ul> Keyboard Layout</h3> teej sets up QMK Layout for Dactyl Manuform 5x7</a></li> </ul> QMK</h3> QMK - Building Firmware</a></li> </ul> Possible Parts</h3> Pro Micro</a></li> </ul> Build Videos</h3> Kevin Eckert - Keyboard Build Stream - Dactyl Manuform</h4> This is great in depth series on the build which matches the types of connectors used in abstracthat/dactyl-manuform</a>. Keyboard Build Stream - Dactyl Manuform - Part 1</a> - assume already printed, put switches in the case, wire the switches</li> Keyboard build Stream - Dactyl Manuform - Part 2</a> - continue wiring switches with diodes on the rows</li> Keyboard build Stream - Dactyl Manuform - Part 3</a> - wire up the Pro Micro to the switches</li> Keyboard Build Stream - Dactyl Manuform - Part 4</a> - Wiring RJ-9, USB-C to micro USB extension, programming mode button</li> </ul> CMD - Dactyl Manuform Build</h4> This build video is great for understanding the background of the Dactyl and seeing someone paint it. However, it is using a slightly different case from abstracthat/dactyl-manuform</a>. It uses a different type of connector to connect the two halves of the keyboard, etc. CMD - Dactyl Manuform Build</a></li> </ul> I built a Ridiculous Keyboard by Hand</h4> I built a Ridiculous Keyboard by Hand</a></li> What I Learned From Building a Ridiculous Keyboard by Hand</a></li> </ul> Keyboard Layout</h4> Developer's 40% Plank layout</a></li> </ul> Dactyl QMK</h3> Getting Started with Dactyl Manuform QMK</a></li> </ul> Wiring / Flashing</h3> QMK Wiring for connect two halves</a></li> Firmware Pin Mapping</a></li> </ul> Split Not Working</h3> Dactyl Manuform Split Not Working</a></li> QMK Split Keyboards Feature</a></li> QMK Split Keyboard Not Working</a></li> </ul> QMK Firmware</h3> Compile</h4> qmk compile -kb handwired/dactyl_manuform/4x5 -km drewdeponte</code></pre>Flash</h4> Connect the keyboard directly to the laptop via straight USB-C cable with the left half plugged into the computer. The right half can stay connected to the left half via the telephone wire.</li> Run the following command, it will get to a point where it starts waiting for the device. When it does, you double tap the button on the back of the left half of the keyboard, and it will switch the controller into flash mode. The app will detect it and flash the device.</li> </ol> qmk flash -kb handwired/dactyl_manuform/4x5 -km drewdeponte</code></pre> After it is done you should be able to disconnect it, reconnect the keyboard and everything should be good to go. Edit</h4> Edit the following file to make changes to the layout. ~/code/personal/qmk_firmware/keyboards/handwired/dactyl_manuform/4x5/keymaps/drewdeponte/keymap.c</code></pre>Notes</h4> To get this split to work I had to set up my config.h</code> as follows: #define USE_SERIAL #define SPLIT_USB_DETECT</code></pre> This only really works when the left side is plugged into the computer. If the right side is plugged in the keys translate differently. I tried the EEPROM</code> detection but even after flashing the EEPROM</code>s it didn't seem to work with the EE_HAND</code>. So after trying things out this is what I found to work. Past Keyboards</h2> I have had many past keyboards. The following is a list of just a few. Kinesis Advantage2 Signature Series</a></li> ErgoDox</a></li> Infinity ErgoDox</a></li> Moonlander</a></li> Planck</a></li> Glove80</a></li> </ul> Git Patch Stack 7.0.0 2024-01-09T00:00:00+00:00 The 7.0.0</code> release of Git Patch Stack is an extremely significant release. It marks the beginning of a few major mental & strategic shifts in terms of Git Patch Stack. State</h2> Historically we have tracked state in all versions of Git Patch Stack before 7.0.0</code>. We stored this state in .git/GIT-PATCH-STACK-PATCH-STATES-V3.json</code>, and a state entry in it generally would look something like the following. { "5543c1e7-b77b-41af-8653-c721b2e9db9d": { "patch_id": "5543c1e7-b77b-41af-8653-c721b2e9db9d", "state": { "Integrated": [ "origin", "ps/rr/fix_grammar___spelling_in_making_code_reusable", "4c4b3b720b92957b21f11a9b6a331190b55535b1" ] } }, }</code></pre> You can conceptually think of it as hash map keyed off of the ps-id</code> (patch_id</code>). The state</code> property is simply an object that kept track of the current state of a given patch. This state</code> property could conceptually be in any of the following cases, each with their own associated data. #[derive(Serialize, Deserialize, Debug, Clone)] pub enum PatchState { BranchCreated(String, String), // branch_name, patch_stack_diff_hash PushedToRemote(String, String, String, String), // remote, branch_name, patch_stack_diff_hash, remote_diff_hash RequestedReview(String, String, String, String), // remote, branch_name, patch_stack_diff_hash, remote_diff_hash Integrated(String, String, String), // remote, branch_name, patch_stack_diff_hash }</code></pre> This was a simple straightforward way of getting started with Git Patch Stack and representing the state that we believed we needed. However, over time we began to see flaws & drawbacks with this approach. Flaws</h3> For one, we got numerous reports of the state tracking becoming out of alignment with Git proper because users were going outside of Git Patch Stack and doing things. Additionally, Git Patch Stack was tracking state that was fabricated. Meaning it was state that we had made up and wasn't actually possible to represent within Git. This is largely because we were caching this fabricated state from remote systems such as GitHub. Creating another point/opportunity to be out of sync in terms of state. Third, people were trying to use Git Patch Stack across multiple remotes of a repository and our state tracking had no mechanism of being synchronized across the remotes. The Decision</h3> With the 7.0.0</code> release we decided to resolve these issues by making a very important decision. Firstly, we would no longer track state. Instead, we would compute the state for Git Patch Stack from the natural state of Git. This has the benefit that it makes it impossible for a users Git state to be out of sync with a users Git Patch Stack state. Beyond that, it facilitates the sharing of Git Patch Stack state across remotes as the Git state is already synchronized over the network. In addition, it has the massive benefit of now enabling users to use only the portions of Git Patch Stack that they find valuable for their workflow without getting out of sync. This same characteristic also enables Git Patch Stack to be combined with other Git extensions & tools (e.g. git-absorb</a>) without worrying about getting out of sync. Beneficial Side Effects</h3> In addition to all the natural benefits we have gained from the decision to compute state instead of track state, we indirectly gained a massive side effect. The side effect of creating better alignment between Git Patch Stack and Git proper. This modeling alignment ended up enabling us to add a long requested feature of Git Patch Stack, Patch Series support. A Patch Series within a Patch Stack workflow is a defined consecutive sequence of patches. Often times they are dependent on one another and have to be sequenced in a specific order. This is useful when you want to simply organize related patches or when you want to request review of a series of patches. Even though we have introduced the concept of a patch series, it is important to realize that they still exist within your Patch Stack. In fact, they can exist alongside other solo patches or other patch series within your stack. Their relationship to each other is simply controlled by which order you place them in your stack. The same way that you manage patch relationships. To understand how we define or refine a patch series lets look at the example above. Prior to having the "login" patch series defined our gps ls</code> output would look as follows. Simply a stack of patches. To define or refine a patch series we simply use our new patch index range syntax (e.g. 2-5</code>), which works generally across the commands. For example if we want to create the patch series locally like we did in this example we would use gps branch -n login 2-5</code>. Leaving us with a gps ls</code> output that again looks as follows. However, if we wanted to refine the definition of that patch series. We would simply rerun that same command while either increasing or decreasing the patch index range to respectively include or exclude patches from the patch series. Alternating Patch Series Coloring</h4> After using Patch Series support for a while within the Git Patch Stack core team, we decided that to really make it easy to use we would need to be able to visually see a patch series within the stack without having to track from the right side of a patch, seeing the branch, all the way over to the left side of the patch to see the index. Our solution for this problem was relatively simple. We introduced alternating patch series colors in the gps list</code> output. This allows you to clearly see which patches are part of which patch series. In fact from the screenshot above you can see that gps list</code> sees individual patches that are not part of a formal patch series as implicit patch series of one patch. This is why the colors are alternating between the Add Mortgage Calculator</code> patch and the Add Loan Calculator</code> patch, as well as the Add Savings Calculator</code> patch. In fact if we had the same set of patches but hadn't defined the patch series we would see the following, as the alternate patch series coloring feature is enabled by default. We can also clearly see the every patch in the list has alternate coloring. This is a side effect of the fact that we don't have any formal patch series defined and again that gps list</code> sees solo patches as a patch series of one patch. Colors and their Problems</h5> Colors especially within the terminal have a number of different problems. For one the contrast needed for different people is different. Two, the colors within my terminal are almost always different from the colors in your terminal. Not to mention how your monitor is configured and presents the colors to you is likely different. Or even, how your eyes interpret the colors and the variance there. Who knows maybe you are color-blind and therefore need specific colors. To address these issues we have shifted to 24-bit color mode and set the default colors based on my personal color choices. This way the colors are hopefully pretty close to the colors I use on my machine. We have also opted to support customization of the colors so that if you need or just want different colors you can configure the colors that are used in the gps list</code> output. For details check out the Configuration</a> documentation. Composition over Opinion</h2> Another characteristic that Git Patch Stack has had historically up until 7.0.0</code> is that it was more opinionated. It expected users to follow a specific workflow and focused primarily on that. This original decision was largely due to the fact that we were tracking state and therefore users had to use our commands & workflow for the tool to consistently work. As part of the original approach we implemented a decent number of commands either for convenience, as an alias, or to try and create a complete abstraction to facilitate having a workflow completely within Git Patch Stack. A lot of these commands actually did nothing other than proxy down to a Git command. This has its maintenance problems as we would then have to maintain and proxy all the various options for that Git command. Git commands change over time and therefore this creates a large maintenance problem. Beyond that it turns out that there are a ton of different variations of workflows that users have with Git. Having a hard lined view of the workflow that Git Patch Stack is targeted to support and not supporting any other workflows significantly limits its ability to be valuable. However, now that we are computing state we no longer need to be so opinionated in terms of workflow as our users can now use the pieces of Git Patch Stack they prefer and ignore the rest. They can also use Git Patch Stack in combination with other Git commands or even other Git extensions. So we decided to change the focus of Git Patch Stack from enforcing a particular workflow, to instead be a general tool allowing users to have their own custom workflows utilizing Git Patch Stack. Making this decision then made it clear that to truly support this mindset we should remove all the "proxy commands". This will ease our maintenance effort significantly, and it will make the intention of Git Patch Stack and its stance clearer. So we removed the sync</code>, log</code>, create-patch</code>, amend-patch</code>, status</code>, unstage</code>, upstream-patches</code>, and the add</code> commands to simplify the command surface area. To continue this effort to simplify the command surface area we removed the batch-request-review</code> command and added proper first class support for batching to the request-review</code> command. When doing so, we added it in a way that would be easy to add to other commands. However, we were hesitant to just add it to all the commands as we weren't sure that it made sense. Additionally, we removed the old branch</code> command and renamed the request-review-branch</code> command to branch</code> as well as retrofitted it to be a first class citizen. We also realized, based on proxy command feature requests, that we needed to somehow support all these custom local Git workflows people have. We decided that the best way to do this was to additionally focus Git Patch Stack on the concept of facilitating composition of its concepts with that of Git proper or any other command line tool for that matter. To begin to facilitate this we have provided the gps sha <patch-index></code> command. It simply takes a patch-index</code> as the argument and outputs the corresponding Git commit SHA. Having this basic command now allows people to build new commands that deal with a patch index instead of a Git commit SHA. Maybe you are a fan of using git commit --fixup</code> and you want to use it with a patch index instead of having to have the SHA for a patch and type it in. No problem with gps sha</code> you can simply do git commit --fixup=amend:$(gps sha 4)</code> to flag patch indexed by 4 for amend. If it is a command you use all the time in your workflow simply make an alias that does it. This flexibility provided by the composition is key to support the vast variety of personalized workflows that people have. Platform Agnostic</h2> Another choice that we made was that we should be as platform-agnostic as possible in our choices and our design so that we can support as many platforms as possible. Support for macOS & Linux variants was something we had at a basic level. However, Windows was a platform that we didn't support primarily because of choices of libraries we used. So part of the effort of the 7.0.0</code> release was to move toward being able to support the Windows platform. As I mentioned before the primary issue we had was libraries we were using that weren't cross-platform. So we swapped those out for libraries that were cross-platform. GPG Signing</h3> The other major issue we had was GPG signing support. Prior to 7.0.0</code> we were integrating directly with GPG via C library that we couldn't easily get to build on Windows. So we decided to follow suite with what Git proper was doing in terms of GPG Signing support and have Git Patch Stack simply shell out to the GPG command on that system by utilizing Git propers configuration. For details on configuring GPG signing please checkout Commit Signing</a> docs. SSH Signing</h3> In addition to switching GPG Signing away from the C library. We also had numerous requests to support SSH Signing as GPG was losing in popularity. So while we were reworking the GPG Signing we also added support for SSH Signing in a cross-platform manner. All of these platform-agnostic choices have also resulted in a far simpler build process which helps not only in the distribution of the application but also in the maintenance and even developer onboarding. For details on configuring SSH signing please checkout Commit Signing</a> docs. Quality of Life</h2> Beyond these major decisions and the improvements that have fallen out of them we also have a lot of improvements that we have made simply for Quality of Life. Integrate Verify Hook</h3> As part of 7.0.0</code> we now have the integrate_verify</code> hook. This is a hook that gets executed as part of the gps integrate</code> commands lifecycle. Enabling you to verify external state, maybe on an SCM hosting provider, is correct prior to integrating. We have provided an example in the repository, example_hooks/integrate_verify.sample.github-cli</a>. This example is written to integrate with GitHub to check to make sure that all the CI checks have passed prior to letting someone integrate their patch. For further details on this hook check out the Hooks</a> documentation. In Rebase Support</h3> For quite some time we have had requests to make gps list</code> function when in the middle of a rebase. Well 7.0.0</code> is the release where we finally add support for this. To see how this works let's get started by setting the state to what you see in the screenshot below. Now let's say that we want to edit the Add Login client</code> patch identified by patch index 3</code>. One approch to doing that is to run a gps rebase</code> and simply mark that patch for edit</code>. This will leave us looking like the following. Git simply outputs the short SHA and summary of the commit it stopped at for editing. However, if we now run gps list</code> we see the following. From the above we can see that it shows us a lot more useful information here. It shows us that we are in the middle of a rebase of main</code> and the SHA of the commit we are rebasing it onto. It also shows us in the bottom section the current state of the patch stack. At this point in time it only consists of the patches that the rebase has already played. From this we can tell that if we were to run a git commit --amend</code> it would be amending the Add Login client</code> patch as it is the patch currently at the top of the stack. Lastly the section above that shows the patches that are still in the rebase to-do list and have yet to be replayed. In fact if you squint a little you might have noticed that if you slammed the top section and bottom section together it would look just like your patch stack. This is by design so that you can quickly and easily figure out what state you are in with respect to your patch stack, even when you are in the middle of a rebase gps list</code>. Ahead & Behind Counts</h3> In addition to all these other great improvements we also improved gps list</code> by adding ahead and behind counts to its output. In fact we can see these on the top line of the gps list</code> output in the screenshot below. In the screenshot above we can see that it says that we are currently in a state where our patch stack is ahead 7</code> and behind 0</code>. What does this mean exactly? Well, ahead X</code> is telling you how many patches stack has ahead of it's base. In the screenshot above it is 7</code> because there are 7</code> patches. The behind Y</code> is telling you how many commits your stack is behind it's associated upstream tracking branch. You generally want it to be 0</code> as that indicates that your patch stack is up-to-date with what Git sees as the current state of your patch stacks upstream tracking branch. You will notice becoming behind if others are adding commits to the remote of your patch stack, and you are using git fetch</code> to update your repositories' knowledge, but you aren't actively rebasing your patch stack onto the latest. Others</h3> Beyond all the above-mentioned features & changes. We have also included a handful of other changes that are less crucial and less visible in terms of presentation. The summary of these can be seen below. Added</h4> conflict & merge commit detection errors to isolate command</li> conflict & merge commit detection errors to branch command</li> conflict & merge commit detection errors to request-review command</li> conflict & merge commit detection errors to integrate command</li> global color option support to the branch command</li> the command alias "b" to the new branch command</li> build time generation of man pages</li> </ul> Changed</h4> unhandled error output for the list command</li> unhandled error printing to print the chain of errors</li> gps --help docs to describe core vs. ancillary cmds</li> improved CLI help interface</li> </ul> Fixed</h4> crash in list command when tracking branch is configured but remote branch doesn't exist</li> bug where commit without diff id in stack would cause list to fail</li> </ul> Where to Go from here?</h2> The future is somewhat an unknown as we haven't planned a next major release yet, let alone a theme for one. To keep tabs on what we are planning or specific issues you can check out our public GitHub Project</a>. If you are interested in chatting about Git Patch Stack or just different Git workflows, providing feedback, or contributing. Don't hesitate to reach out via our public Slack Group</a>. If you are completely new to Git Patch Stack, a good place to start would be the main Git Patch Stack site</a> as well as the documentation</a>. As always we hope that you find the decisions, changes and features we added valuable. Building a StoryBrand: Chapter 4 2023-12-28T00:00:00+00:00 The content goes here. Context</h2> Principle One: The customer is the hero not your brand. Define characters ambition. What they want. So that we open the question, Will the hero get what she wants? This is called a Story Question or story gap. When we define something our customers want, we posit a story question in the mind of the customer: Can this brand really help me get what I want? define something your customers want</li> must be have a singular focus, be one thing</li> </ul> Example 1</h3> High-end resort originally had marketing content that was images of their front desk, staff, etc. Wasn't really inviting a customer into a story. What their customers really wanted most was a luxurious, restful experience. </blockquote> After StoryBranding they changed their website from long stories about themselves (which positioned them as the hero) to images of a warm bath, plush towels and robes, someone getting a massage in the spa, and a looping clip of a back-porch rocking chair against the backdrop of trees blowing in the wind along a golf course. </blockquote> They replaced the text on their main page with the short and powerful copy: "Find the luxury and rest you've been looking for." </blockquote> Excerpt From Building a StoryBrand Donald Miller https://books.apple.com/us/book/building-a-storybrand/id1212150000 This material may be protected by copyright. Example 2</h3> "One university we worked with defined their customer's desire as “a hassle-free MBA you can complete after work." </blockquote> Excerpt From Building a StoryBrand Donald Miller https://books.apple.com/us/book/building-a-storybrand/id1212150000 This material may be protected by copyright. Example 3</h3> "A landscaping company humorously defined their customer’s ambition as “a yard that looks better than your neighbor's." </blockquote> Excerpt From Building a StoryBrand Donald Miller https://books.apple.com/us/book/building-a-storybrand/id1212150000 This material may be protected by copyright. Example 4</h3> "A caterer we worked with in Los Angeles defined his customer's desire as “a mobile fine-dining experience in the environment of your choice." </blockquote> Excerpt From Building a StoryBrand Donald Miller https://books.apple.com/us/book/building-a-storybrand/id1212150000 This material may be protected by copyright. Examples 5-10</h3> Financial Advisor: "A Plan for Your Retirement" College Alumni Association: "Leave a Meaningful Legacy" Fine-Dining Restaurant: "A Meal Everybody Will Remember" Real Estate Agent: "The Home You’ve Dreamed About" Bookstore: "A Story to Get Lost In" Breakfast Bars: "A Healthy Start to Your Day" </blockquote> Excerpt From Building a StoryBrand Donald Miller https://books.apple.com/us/book/building-a-storybrand/id1212150000 This material may be protected by copyright. Actions</h2> Application</h2> Getting Started with StoryBranding 2023-12-28T00:00:00+00:00 If I am honest with myself. One of the areas that I lack strength & skills, is marketing. I try and focus on areas I am weaker to hopefully improve over time. But it is hard to do this when you are running a business, doing client work, and have a family. A while back a friend of mine introduced me to the book, Building a StoryBrand</a> by Donald Miller. I read it through, and it resonated pretty strongly with me. But it was non-trivial for me to apply. So I let other things get in the way. I feel like if I had better marketing skills it would seriously impact my ability to help push my business to the next level. So being that I have some time during Christmas vacation I have decided to give it another go. This time pushing myself through the discomfort in hopes to grow my skills & abilities. I figure if I can at least start it during Christmas break and keep a journal as I go through then I will be able to actually push through and finish with something to show for it. So jumping right in, Chapters 1, 2, and 3 are really an outline and sales pitch for the concept of StoryBranding and the framework to create a StoryBrand. StoryBranding is basically the idea of taking the same fundamental structures that make stories so impactful to people and applying them to your branding. The later chapters get into specific details about the framework and what each step is. I will cover those in detail in future posts as I reread them and attempt to apply them. Right now I am planning on using a rough template consisting of two main sections. context</li> actions</li> application</li> </ul> The thinking behind this is that I am focusing on details of how to apply this concept which I will outline in the actions section. While the context section will house all my notes from reading the chapter that I believe are necessary context while doing the actions. The application section will house an explanation of how I have attempted to execute the actions. I have a number of different projects, services, etc. that I could apply this concept to that I may use some of in the application section just to try and get a better handle of things. Pullwalla</a></li> Git Patch Stack</a></li> Uptech Studio</a></li> Uptech Studio as a Productized Service</li> </ul> Pullwalla - More Workflows 2023-11-28T00:00:00+00:00 Hello! We just released v3.7.0 (51) of Pullwalla</a> to both macOS and iOS platforms. Quick Filters</h2> One of Pullwalla's primary goals has been to help people quickly discover the pull requests they are interested in. Quick Filters continue to be a key mechanism supporting this. More Workflows</h3> One of the things we have done in this release is support additional workflows. The workflow we supported historically is a workflow where developers take a step back from code every so often, looked at the previously Unclaimed, pull requests to find a pull request review, and then "claim" it by assigning themselves to it. This workflow is fine and works for some. But it doesn't work for everyone. Some people instead take the approach of requiring that every pull request when created has to have explicit requests for review, or requires that assignees be used to mean something different. Others have a workflow that is some sort of hybrid between the two approaches. Therefore, we introduced two new Quick Filters. Undecided - shows pull requests that have not been approved or had changes requested yet.</li> Needs my Decision - shows pull requests that either you are an assignee of or that have a request for your review and have not been approved or had changes requested by you.</li> </ul> The Needs my Decision quick filter facilitates workflows where PRs are created and given an assignee or a request for review is explicitly made. While the Undecided quick filter supports workflows where developers pick their own pull requests to review. You can see this in the screenshot below. Clarification</h3> Beyond the addition of the Undecided and Needs my Decision quick filters. We also renamed the other quick filters to try and make them more accurate and clear. Previously Pullwalla's Quick Filters looked as follows. The mapping of the name changes are as follows. Created is now My PRs as it shows pull requests that you have created.</li> Assigned is now My Assignments as it shows pull requests that you are an assignee of.</li> Mentioned is now My Mentions as it shows pull requests that you are mentioned in.</li> Review Requests is now Request for my Review as it shows pull requests where someone has requested your review.</li> Participated is now Participated In as it shows pull requests that you have participated in.</li> Unclaimed is now Unassigned as it shows you pull requests that have no assignees.</li> Claimed is now Assigned as it shows you pull requests that have one or more assignees.</li> </ul> We hope these improvements to Quick Filters will have a big impact on peoples day to day interactions with Pullwalla. If you think we are missing something or have any feedback in general please don't hesitate to reach out at pullwalla@uptechstudio.com</a>. Thoughts or suggestions?</h3> As always, we’d love to hear from you. Need help with an App (mobile, web, desktop, etc.)? Uptech Studio</a>, the creators of Pullwalla, are here to help. Just reach out to info@upechstudio.com</a>, and we are happy to explore how we might work together. Cheers, The Pullwalla Team Git Patch Stack Goals 2023-11-03T00:00:00+00:00 The following is a transcript of an audio recording I made exploring the goals and intent behind the Git Patch Stack project. I have also included the audio below if you would prefer to listen. Your browser does not support the audio element. </audio> Transcript</h2> Hi I am Drew De Ponte, Partner at Uptech Studio & creator of Git Patch Stack. Today I wanted to talk about Git Patch Stack and the driving forces and purpose behind it. Largely in my mind, the goal was and still is to find alignment between Git itself as a tool, its characteristics, best practices and the development process and tooling that we all generally use at mass today. In my mind that is using Bitbucket, GitHub, GitLab and the pull requests or merge requests peer review workflow that the mass is still using today. The reason I was looking for alignment between these things was because I felt like there was a dichotomy between the pull request feature branch workflow and Git best practices. How people were using feature branches in the feature branch workflow at large was very much against the best practices of Git. One great example of this is the atomic or logical commit being a best practice of Git. This is because people would focus largely on feature branches as the primary concept and they would ignore the value of commits. They would ignore the benefits you gain longer term by having well structured logical commits, and they would ignore the benefits you get in terms of all the features of Git being available to you. So Git Patch Stack has always been an exploration on my part to try and find the best alignment between those two worlds. Most of our values have fallen out of this exploration for this alignment. For example one of our values is that we support the Continuous Integration Methodology as closely as possible. We want to eliminate the negative side effects of branch isolation. So, We always lean towards Continous Integration in terms of decision making. Having the user confront conflicts and bring integration to the forefront so that it happens at a much quicker iterative pace, and so that the surface area of the conflicts are much smaller when they occur. Rather than waiting until a giant feature branch is finally ready to be merged and having to deal with a nightmare in terms of integration. We are also big on aligning with Git. Hence why we focus on atomic or logical commits and the tooling is designed around it. Not that you can't use branches, because you can if you really want. Branches are fine. But you have to be a very cognizant and aware of the consequences of those branches when you are makeing those decisions. I think that's largely one of the downfalls that occurred when Git came out. Git itself is fantastic as a tool. It gives you all this flexibility and power. However, when GitHub came out as a tool it created this next level of focus on the concepts of the feature branch & pull request. Which only perpetuated heavy use branches and drove developers to not think about the concept of logical commits. Or even the fundamentals of Git anymore. It's a higher level abstraction that they created & popularized. And most developers nowadays come into Git through these higher level abstractions. I can't tell you how many developers I've met that don't even know that there is a tree of commits. And that's a side effect of them learning from this higher level abstraction and not learning the fundamental underlying abstractions. There's downsides to that. Which is, without diving deeper and learning those underlying abstractions, those developers just believe that, that's the truth. They believe that, that is the only way. Or, that in actuality, they believe that, that is the way it was designed and intended to be used, which is not necessarily true. That's one person or one group of people's take on how they think it should be used. And none of those people were the creator of Git. They built that and put it out into the world and it resonated with people. This makes sense because it's a simplification. Instead of thinking about commits and how they relate to each other & all the complexities of the mechanics of managing them in Git. You don't have to think about any of that. You just think about this much grander, larger scale thing called a feature. But that has its downsides as well. So again, we as Git Patch Stack are focused on trying to find & and refine that alignment between these worlds, where we can focus on the right things for Git as a tool itself. But also play in a world with these higher level concepts like feature branches & pull requsets, when we want or need to. And also allow the user to, jump out and use a branch or any other normal Git commads when it is appropriate. But the focus of the tooling is all on these logical commits. So we have this core value of, abide by Git best practices. That way we are in alignment with what Git expects. Support as much as possible the concept of continuous integration. So that we can reduce the surface area and complexity of conflicts. Third we have, finding alignment with this concept of feature branches and pull requests and peer reviews. But implementing it in such a way, that you aren't forced into the feature branch, pull request, or peer review concepts. Instead, Git Patch Stack allows you from a workflow standpoint to plug into those concepts, or it would allow you to, plug into an email based workflow instead. When you conceptually request review in Git Patch Stack, there's a hook system. And that hook system allows you to plug in a hook to, go create a pull request in GitHub, or go create a patch and email that patch to a particular mailing list, or do anything you want. It's completely up to you. But the goal of Git Patch Stack is to provide that core structure to think in terms of the right concepts so that you are in alignment with all of these things, not just GitHub's model. Another value, is being able to focus on how to mentally think about these things and trying to do that with as little overhead as possible. You can see this point in terms of a comparison of the Stacked Branch workflow versus a Patch Stack workflow. In a Stacked Branch workflow, you are creating branches for each of your atomic commits, and then you're defining the relationships between those branches so it can formally store a directed acyclic graph of those relationships. And then there's extra tooling both on the client side, where you manage your changes locally, but also tooling on the application side where it can read the directed acyclic graph and facilitate reviewing pull requests in the particular order defined in the directed acyclic graph. The DAG also enables additional automation in terms of management of these branches and the DAG. In my opinion, the Stacked Branch workflow's functionally solves some of the problems that you have when you start thinking about atomic commits. But it doesn't solve all of them. And it requires this overhead of defining and managing the branches & the DAG in order to even solve the problems it does. Again my goal with Git Patch Stack was to explore and find a way that things fall into alignment with all of these things. That way developers can just focus on the core concepts and principles & naturally be in alignment. For example ideally, you don't want to have to worry about creating branches up front and maintaining branches at all. Instead, the relationships of your patches are managed through their order in your patch stack. And it's much easier to think about those dependencies because that is the way those dependencies end up in your actual tree later on. The last value that I can think that we focus on is, and this isn't that you're forced to do this but we do encourage it, that you have a linear Git history. If you think about it, the concept of a patch stack is a linear history itself. And so the idea is that when you integrate, if you're integrating through Git Patch Stacks tooling, you will end up with a linear history. Now, technically Git Patch Stack is flexible enough that you can integrate through GitHub, Bitbucket, or GitLab merges. If you do we recommend that you configure it to do a rebase and merge so that you end up with a linear git history. But from a technical standpoint, you don't have to integrate that way. You could do forced merges where you always have a merge commit, for example, where you use git merge --no-ff</code>. It'll always create a commit even if it could fast forward. Some people prefer to do that. Long ago in my career I believed that was the best approach because I was focused on feature branches back then. However, now I no longer agree with it. So instead we push this idea of linear Git history. This helps all kinds of things down the road. It helps you understand your Git history more quickly and easily. It helps you understand and be able to revert logical changes from your Git history. I hope that provides some insight into what the goals are and why Git Patch Stack exists. It's definitely continues to be a journey exploring and trying to find, or now days more like, refine this alignment. Git Workflows 2023-10-26T00:00:00+00:00 The following is a transcript of an audio recording I made exploring some of the benefits and drawbacks of the Feature Branch, Classic Continuous Integration, Stacked Branch, and Patch Stack workflows. I have also included the audio below if you would prefer to listen. Your browser does not support the audio element. </audio> Note: I added some headers into the transcript to try and make it a tad bit easier to jump to different sections if you are particularly interested in one workflow. Transcript</h2> All right. So let's talk about some common git workflows. Feature Branch Workflow</h3> One of the most common git workflows is the Feature Branching workflow. And I think one of the primary reasons why it's so common, is largely due to the fact that GitHub was created and GitHub heavily promoted this workflow. This of course is because GitHub is built around this workflows model. It is worth noting, that just because it's the most common doesn't mean it is the best workflow. It doesn't mean it's the worst workflow. It could be somewhere in between. The truth is the commonality of it doesn't necessarily indicate its value, quality, applicability to our needs. We don't know until we actually reason about it. Comparing and contrasting the benefits and drawbacks of these various workflows. And even then, we still might not know definitively. But it will likely give us a bit more confidence and definitely a deeper understanding. So let's get into the Feature Branch workflow. The way this works is that you have your git tree, and you have a main branch, which is your main line of development. Now, you don't work directly on that branch. Instead, what you do is you create a feature branch based on it, to do your work in. And so in that feature branch, you make, all of your commits, all of your changes, inside of that feature branch which is isolated away from all of the code in the main line that's coming in from other developers. So you're in this safe little isolated world where you can pretend that no other changes are really happening for some period of time. Basically the period of time that it takes you to develop that feature. And then you have to work to get that feature branch merged back into the main branch and pushed up to the upstream's main. OK. Now how you go about this depends on your team dynamics and structure. The way github facilitates this is via this concept they introduced called a Pull Request. To use it you take your feature branch, you push your feature branch up to the remote as a separate branch and then you open a Pull Request which is effectively a merge request asking one of your teammates to review & approve the merge of that branch into main line. This creates a Pull Request on GitHub. And you can see the diff between the main line and the Feature Branch. So you can see the changes made and people can comment on them and you can discuss and debate whether those changes are good and you can iterate on your feature and so on and so forth. And then eventually, you know, once that's approved and good, then you can merge that into main, and you can push main up to the upstream main. GitHub helps kind of streamline the merging process a little bit by providing a button to trigger the merge via the web interface. OK. Now, so far, we've talked about a few things. There's a couple of things that are nice and a couple of things that are not nice. One of the things we've talked about is this convenient green button merge thing. We talked about this idea of a Pull Request where you can see the changes and that's also kind of nice. But the other thing we talked about was this branch and this idea of branch isolation. Which might seem like a nice thing initially, but you soon realize that the longer you have a branch in isolation and you pretend that there aren't other changes being made to the repository from other teammates, you are increasing the probability that you will run into a conflict and you are increasing the probability that the conflict when you run into, will be larger and more complicated to resolve. So that is a downside to this approach. The other downside that's kind of hidden to this approach. Or atleast less obvious, is that Git as a tool promotes this concept of having logical small commits. OK. And logical commits means that they are a unit in themselves. Another term related to this is called atomic commits. Which can be conceptually thought of as the smallest, atom like commit, or indivisible logical unit. But fundamentally, the idea is that it's a small logical change that that doesn't depend on other changes above it? It only depends on changes that already exist in the git tree. It's a logical change that is buildable and testable and everything the way you want it. And that commits in general should be designed in this way. So the Feature Branch workflow with GitHub has kind of led away from this idea. What generally happens is that people don't take the time or think about the commits themselves how they should be logically chunked. In general people using the Feature Branch workflow don't tend to think much about the commits at all. Instead the developers tend to just think about the Feature Branch as this logically chunked thing and they don't worry about the state of the commits or whether they're logically chunked or not. Now, the reason this is a problem is because the Git tool itself, and the tooling that it provides is designed around these commits being smaller logical commits. So Git has features that are built around this concept logical commits, and if you don't treat your commits that way, these features don't really work. So one of those super useful features is called git bisect. It is extremely useful when you're trying to isolate a bug by narrow down what code that introduced it. But there are other commands as well. Even when you do something like a git blame and you're trying to understand why a change was was made. Having logical commits is invaluable. So the downsides of this Feature Branch workflow are that it kind of takes the focus off of the commit and moves the focus strictly on to this idea of a Feature Branch. And therefore people naturally pay less attention to the commits. Resulting in you losing the ability to use those vuluable tools. Then we also have the other downside of it, which is the branch isolation. When you have a branch isolated for that long, you increase the probability of conflicts and the difficulty of those conflicts. OK. So that's the Feature Branch workflow. Classic Continuous Integration Workflow</h3> So let's talk about this other workflow I will call the Classic Continuous Integration workflow. The reason I call this the Classic Continuous Integration workflow is because it is largely a workflow that people do when following the Continuous Integration methodology. It is a bit simpler than the branch based workflow because it doesn't really deal with branches. That doesn't mean you can't do branches with it if you want. But in general you don't. Instead, what you do is you make commits directly on your local main branch and you pull in changes from upstream on a regular basis. Now when you pull these changes in you do it in such a way that youl local main branch is rebased onto the latest changes from upstream's main. Once you get your commit on local main to a good place you integrate your commit with upstream's main by pushing it up. So it's much simpler and it lets you focus on the commits. Which makes it easier to create logical commits and think about the commits. So we are in alignment with the Git and their best practices and expectations to support functionality within their tooling. The reason it's called continuous integration is because this workflow is driven off of people following the continuous integration methodology. Where the idea is that you pull in changes on a regular basis and that you are integrating your changes with upstream on a regular basis. When we say a regular basis, we generally mean as quickly as possible. This is because the the purest idea of continuous integration is that you and your team are always working off of a shared single state. Now, in reality, with the tooling today that's not completely possible. But with the idea is that with continuous integration, you want to get as close to that ideal as possible. And so we have this, via this constant pulling and making a commit and integrating it upstream and then pulling, making a commit, integrating it upstream, etc. is fundamentally the flow that you have. Now there this solved some problems, right? For example we're now focused on the commit, the logical commit. So we don't have that problem, we don't have branches. We don't have to worry about branch isolation, which reduces the odds of conflicts and their respective difficulty. This is because we're working closer to that shared concept of shared truth. And all of that's great, but it does have its drawbacks, right? The rub comes when you're dealing with creating these logical commits on top of main. It requires you to do one of two things, to either write your code from an inside out perspective. Meaning you have to guess what's going to be needed as a dependency to build the future feature you're trying to build and go do that inner most dependency first, hoping that you've written its interface correctly and then integrate that into upstream. And then the layer above it, and so on and so forth, eventually to get to where your feature is complete. But just hoping that your interface is correct, in my experience, is very difficult. It is not very comfortable because you don't have confidence that it's correct because you have no other software driving that development or given you feedback about its correctness. So that's one approach you can take, but that's very difficult. The other approach you can take is to use a technique called Feature Toggling, which allows you to integrate code into upstream that is incomplete. So you can write code that is not complete, that is not currently being used because you have a feature toggle that's turned off. But you can integrate that code so that your other developers on your team are aware of it and know about it and could build on it, provide feedback to you, or provide guidance about how it might be used in another case, etc. All of which are positive, useful things. However, managing feature toggles is another layer of complexity, that has to be dealt with. Now managing that layer of complexity has a lot of other benefits. Specifically Feature Toggles end up having a bunch of other benefits when you start talking about things like release management. So there's a ton of other benefits that you get that in my opinion are much more worth the overhead than the complexity of branches which don't help you in those realms at all. Branches only really help you with organization at the workflow layer, but they don't help you beyond that. One of the other side effects of this workflow, is that the peer review process is very different. The idea is that instead of opening a pull request and having your code reviewed, you do post commit review, which is where you have committed and integrated your commit upstream prior to any review by a peer. And you're saying that is OK because it passes the build, and it passes all the tests. It is also been integrated into main and our automated CI tests have not failed. This is very different from the Feature Branch workflow where the review happens prior to integration into upstream main. Because in the Classic Continuous Integration workflow, the code gets integrated into main prior to being reviewed by other developers. You might be going well. This is a horrible idea. How could we do that? The main is supposed to be this pure, perfect pristine thing. The justification for this is simply the realization that you are lying to yourself if you actually believe any code is perfect before it goes in. Just think of how many times you have integrated a reviewed feature and then found a bug or realized that it needed to be refactored to support something else. The Classic Continuous Integration stance is simply accepting that main is this ever evolving and changing thing and therefore adjusting it with a future commit is no big deal as that is how we normally change it. Now, if you talk about tooling Feature Branch workflow works great with GitHub, Bitbucket, and GitLab. As they are all built around the same model. But when you start talking about the continuous integration methodology, and you start talking about this post commit review process, it's really difficult to find off the shelf tools out there that focus on post commit review process Most of them are kind of homegrown tools. But yeah, so, that's the Classic Continuous Integration workflow. Stacked Branch Workflow</h3> This brings us to the Stacked Branch Workflow. The Stacked Branch workflow is trying to solve one of the complexities that I talked about in the Classic Continuous Integration workflow while keeping the focus on these atomic logical commits. So if you remember, one of the hidden complexities I mentioned was that when you're doing the Classic Continuous Integration workflow, you have to develop your code from the inside out, or you have to use feature toggles. So the way that the Stacked Branches workflow works is that you create a branch, but each of your branches are really focused on one of these atomic logical commits conceptually. You're making a branch for each of these tiny logical changes that you're making and then there's additional tooling on top of Git that creates this directed acyclic graph (DAG) that understands the relationship between your branches. This allows you to create branches and then define the dependency relationships between them, effectively building a stack (or sequence) of related (or dependent) branches. Having this directed acyclic graph and tooling that is aware of it enables the tooling to automate what would be an extremely tedious process of rebasing all the following branches when a lower level dependency branch changes. It also gives other tooling the information to know in which order the branches need to be integrated. OK. So it's trying to focus on these logical commits. It's not totally in alignment with the continuous integration methodology. But it does facilitate when you pull, rebasing all of your stack of branches on top of the latest main. So you're at least locally integrating more frequently. However, it fixes these problems by adding all of these layers of extra work. Having to manage creating of branches, having to define relationships between branches, having to rebase branches, etc. It does enable you to build your code from the outside in though, which is good. You still have to integrate your code in an inside out fashion or use feature toggles. The other downside to this approach is that because this isn't part of pure Git. And it does not align with the current Feature Branch workflow of GitHub, Bitbucket, or GitLab. You need additional tooling on top of that. So your entire team has to switch to this additional tooling and this additional application to see the branches, code review, and know the order in which those branches need to be reviewed, etc. OK. So that's the other major downside that even though it starts to solve one of the core issues with the Classic Continuous Integration workflow. Patch Stack</h3> So the last workflow that I'll talk about is what we call the Patch Stack workflow. As you might imagine from the name, this workflow is model around the concept of managing a stack of patches. Now this isn't a stack in the purest sense, you aren't limited to just push & pop operations. Like you might think if you're thinking about a stack data structure. You can reorder patches in the stack, put things in the middle of the stack, pull things out of the stack, etc. You can imagine it's just this mutable stack of patches that you can change and mutate and do whatever you want with. In this flow, you have a stack of patches which, are basically the commits that you make on top of your local main that have not yet been integrated into upstream main. And when you do the get pull, it rebases your local stack of patches onto the now updated origin/main. So this meets the continuous integration needs because anytime you do a Git pull, it is forcing you to integrate your local stack with the changes that have been introduced by your other teammates. Not only that, it focuses on this concept of patches or commits, which are just logical commits. These are the same small logical commits, which are also in alignment with continuous integration. Beyond that it allows you to develop your software in an outside-in fashion while still doing the inside out integration similar to the stacked branch workflow. Except in this workflow, you don't really concern yourself with branches at all. You don't deal with the overhead of branches. Instead, you think about a stack of patches. And if you want a commit to exist, that depends on another commit in your stack, you just have to make sure that the commit that it depends on is lower in the stack than the commit that you're working on. This allows you to refine patches over time and work outside in. But as you resolve and figure out what those inner most dependencies are, you can integrate those right away like you would in a continuous integration workflow or you could request review for them earlier to get them reviewed and integrated into the code base as quickly as possible, which is also more in alignment with the continuous integration methodology. That sounds great. It sounds like it's solving all the problems of the continuous integration workflow. So let's talk about peer reviews though. The Patch Stack workflow allows you to request review of an individual patch or a series of patches. And what happens behind the scenes is that this actually ends up mapping back to branches. When you request review of an individual patch, behind the scenes, it goes and creates the branch, cherry-picks that patch into that branch, pushes that branch up to the remote, and can even open the pull request for you on GitHub, Bitbucket, etc. This makes it so, it works with the existing tooling that's there. In turn allowing you your team to use the same workflow they've always used and the same tooling they've already been using. But locally, you can develop your code using the Patch Stack workflow. You might be asking yourself, wait, what about this dependency thing? Do we have deal with all the complexity of managing branches locally and then defining the relationships between those branches to define the relationships between dependencies? Nope, the Patch Stack workflow takes a different stance. When you have dependent commits like that you have two options. One is you request review for those individual patches one at a time in the order that they need to be integrated and wait for the core dependent one to be integrated before you request review of the next one. Or you can create what's called a patch series and request review of a patch series. All that is, is saying, hey, I want to request review for these three patches in series in this order they exist in the stack. And that does the same thing behind the scenes. It'll create a branch, it'll cherry-pick those patches in sequence into that branch, push it up to the remote, and create a pull request for you based on those patches going into upstream main. So that's how the Patch Stack workflow deals with dependent patches. You can have multiple patch series stacked on top of each other in your patch stack. You can also have individual patches inter weaved in there if you want. It allows you to manage all of that directly within your patch stack, which is just you focusing and thinking about logical patches. So that's the Patch Stack workflow. The benefits of the Patch Stack workflow is that it gets you as close as possible to continuous integration but gives you the flexibility of defining a patch series if you need to. So you don't have that overhead of being required to manage these branches and their dependencies between each other. Beyond that, it works with the existing tooling. So it works with the normal GitHub, Bitbucket, GitLab, etc. and their Feature Branch based review process. So you don't have to worry about any of this post commit review stuff, or the tooling around that, or getting custom tooling made, or any of that stuff. You just use the same tools you are likely already using. In fact, another benefit of this is that it facilitates adoption more easily. So let's say you as an individual are impressed with what you've seen from the atomic commit community and want to start adopting those practices. But your team is still working in a Feature Branch workflow. Well, that's completely fine. You can use the Patch Stack workflow locally and the rest of your team never even has to know. They don't care, they just see a Pull Request, and they do a review, the same way they have previously for the Feature Branch workflow. Conclusion</h3> So those are the four different workflows I wanted to talk about. Hopefully, this helps clear up some of the concepts as well as some of the pros and cons of these four workflows. Personally, I think the Patch Stack workflow makes the most sense. Anyway, hope you enjoyed. Until next time. Cursor based Pagination with multi-key cursor 2023-04-02T00:00:00+00:00 Let's say for sake of discussion that we want to paginate a set of transactions to be presented in a mobile app with infinite scroll implemented. We will also assume that for sake of discussion our service is backed by PostgreSQL as our database. We obviously want to see these transactions in relation to time. Probably the most recently created transactions first and then the older transactions as we scroll through the infinite scroll. So we will likely want to order these transactions by their timestamp in descending order. Cursor based Pagination</h2> Because of our use case we can use a strategy called Cursor Based pagination. In this strategy we pass a "cursor" identifying the transaction of the last item in the previous pages results and a limit to get back the next page. Use transaction_datetime</code> as cursor</h2> You might be thinking, well we have the transaction_datetime</code>, can't we just use that as the "cursor". The answer is of course you could. However, if you have timestamps that aren't super granular. This happens when dealing with banks & transactions as you will get back a full timestamp object, but it will only have information for the day, month, and year. This means in your dataset you may have a decent number of transactions that are all effectively at the same timestamp. This is problematic because you have to either do a where condition in your query of <</code> or <=</code> to the cursor. If we do <</code> we will miss out on transactions that should be visible. If we do <=</code> we will end up with duplicate transactions. Neither of these are acceptable from a user experience standpoint. What if we combine transaction_datetime</code> with another cursor</h2> Well if we for example had a UUID for the id</code> column of a transaction we could use that, as ordering of UUIDs would be consistent. This would allow us to have a query like the following. SELECT * FROM transactions ORDER BY transaction_datetime DESC, id DESC;</code></pre> In the above we use a primary order of transaction_datetime</code> with a secondary order of id</code>. This makes it so that if there are multiple transactions associated with the same transaction_datetime</code> that they will all be consistently ordered by the id</code>. Getting the Next Page</h2> Somehow we have to use the cursor now composed of two values transaction_datetime</code> and id</code> to figure out where the dataset should start for the next page of values. If we tried to do something like the following SELECT * FROM transactions WHERE transaction_datetime < '2021-02-23 04:23:23' AND id < 'A234U-23432aue-2343aue-aauo234'</code></pre> We would have a problem because we would be filtering out transactions that are less than the datetime but greater than the UUID. In turn making the resulting set not what we want. So instead we have to use a feature of PostgreSQL called Composite Values in the where query. I think of this as allowing us to create a tuple of values that it then knows how to compare in combination rather than individually. SELECT * FROM transactions WHERE (transaction_datetime, id) < ('2021-02-23 04:23:23', 'A234U-23432aue-2343aue-aauo234')</code></pre> With this we actually get back the correct set as we are sorting on the combined value not the individual values. Inversion of Control 2023-02-09T00:00:00+00:00 Within the last month, I have interacted with several developers that don't have a great understanding of the concept of Inversion of Control (IoC). Just the other day one of the developers at Uptech Studio</a> asked me to explain it after I mentioned it as a way of separating DB concerns from business logic concerns when the DB concerns are tied together by a DB transaction. I introduced this developer to the concept of IoC and walked him through how I think about code and the concept of Separation of Concerns in a way that enables me to more easily understand how and where to apply IoC. So I figured I might as well write it up so that I can share it with the rest of the world in hopes that it can help others as well. So let's get into it. What Is Inversion of Control?</h2> The other day I was talking to another developer about struggling to find a good resource that really explained IoC, and half-jokingly he asked ChatGPT to explain it with the following. Can you explain the coding principle “inversion of control”? </blockquote> It came back with the following. Inversion of control (IoC) is a software design principle that encourages the design of loosely coupled components in which the flow of control is not determined by a component itself, but by a container or framework that is responsible for assembling the components and coordinating their behavior. This results in a separation of concerns between the components and the container, where the components focus on their own specific tasks, while the container is responsible for managing their interactions. IoC is often achieved through dependency injection, in which the container supplies the components with their dependencies, rather than the components creating or looking up their own dependencies. This leads to a flexible and modular architecture, where components can be easily replaced or reused without affecting the rest of the system. </blockquote> To be honest it did a great job at returning a pretty clear definition of what IoC is. Good enough in fact that I wouldn't even try to change it. Trying to get ChatGPT to provide meaningful examples in article form, that I could hand someone to teach them IoC, was a massive failure. And as we all know being able to read a conceptual definition like the one above and being able to understand it, internalize it, and apply it while coding in the wild is an entirely different thing. Hence, why I am writing this article. IoC at all the levels</h2> IoC as a concept is applied & can be applied across all the different levels of abstraction in programming. At the higher levels, we can classically see it with the distinction between the concept of libraries and frameworks. Frameworks are simply libraries that apply the concept of IoC. If we consider how we use a GUI framework. We generally create some sort of Application level object and call run()</code> on it. The GUI framework is in control of the flow of execution of the program and reaches out to other objects, we provide, to ask them to perform different pieces and hand the framework back results so that it can continue to manage the control flow of the application. On the other hand, if we were using a GUI library. We would be having to manage the control flow of the application ourselves, and we would in turn be calling out to different functions/methods in the library to do different pieces for us. So with this distinction, we can see that in the library case we are responsible for the control flow of the application but in the framework case we hand that responsibility of control flow over to the framework. At the lower levels of abstraction, we classically see it with most higher-order functions, especially the higher-order functions on collections, e.g. map</code>, reduce</code>, filter</code>, etc. If we consider using filter</code> as follows. someCollection.filter((item) => item.isOld)</code></pre> We are handing the control flow of the program over to the filter()</code> procedure while it asks us to provide criteria to filter with via the provided closure. This is an example of IoC just as the framework vs. library concept is. The big distinction between them is that in the case of a full application framework you are handing over control for the lifetime of the application. In the case of filter()</code> we are handing over control only for the lifetime of that particular application of the filter process. The key is understanding that we can hand over the control flow of the application at any of the levels of abstraction and it is all IoC. It is inverting the control by handing it over to something that in classical imperative procedural code does not have the "control". Understanding IoC with Code</h2> To better understand how to think about this concept in relation to code let's take a look at a simple example where we have a collection of numbers and we want to have a new collection of numbers that has each value in it but with 5 added to it. In classic imperative procedural code, we would implement this something like the following. const myNumbers: Array<number> = [2, 4, 7, 13] let myNewNumbers: Array<number> = [] for (num in myNumbers) { let newNum = num + 5 myNewNumbers.push(newNum) } // myNewNumbers is now [7, 9, 12, 18]</code></pre>Separation of Concerns</h3> Before we can start to think about the possibility of applying the concept of IoC we have some groundwork that we have to do first. We need to understand the various concerns/responsibilities involved in the code and classify the various lines, statements, and expressions in terms of these concerns/responsibilities. It is also extremely useful to understand the high-level goal of the segment of code being analyzed as it will help form our thinking conceptually when applying IoC. It is probably worth talking about the idea of a concern, responsibility, domain, or problem space which are mostly used interchangeably. They all are used to define a particular bound around a problem. The difficulty people find with them is that you can in theory take a bound around any problem and break it down further into smaller bounds around sub-problems infinitely. The other thing that people tend to find difficult is that they are arbitrary. Meaning you can define the bounds to be whatever you want. In practice in software development, this is simply defining a bound around a problem space and generally mapping an abstraction of some sort to that bound. This breaking up of a bound around a problem into sub-problems with their bounds is the foundation of the concept of Separation of Concerns. And as we will see throughout this process IoC is simply a specific example of Separation of Concerns. Understanding the Concerns</h3> Let's take a look at the example from above and see if we can identify the overall concern/responsibility and then try to classify concerns that the various lines of code belong to. Overall the concern of the example above is to take the myNumbers</code> array and create a new array containing the values from myNumbers</code> but with 5 added to it. That doesn't feel like much of a concern, but technically it is and since it is what we have right now let's run with it. Now that we feel like we have a high-level concern figured out let's think about IoC a bit deeper. We talked at a high level about what the inversion portion of IoC is. But what is the control portion in our current context. The way I think about it is by trying to break the overall concern into sub concerns where one of the sub concerns is the "control". To do that we need to understand what the "control" is. It is essentially any portion of the code that controls the flow of a process. So we have to start thinking about the code that controls the flow of a process as one concern and the various steps/pieces of the process as other concerns. I also think about this as code representing one of two things, the triggering of doing something & the actual how of doing something. So we can start thinking about the ability to separate the how of doing something from the triggering of doing something. It may not be a perfect way of thinking, but it has tended to help me. Let's try and define the various concerns of our example. It is important to realize that this is a process. It isn't magical, and you don't always know all the concerns right out of the gate. Sometimes you just start knowing that one concern is the "control" and the others you aren't sure about yet. Removing the noise of the code that you know is the "control" can help you think more clearly about the code that is not control and its respective concerns. Below is our example from above in classical imperative procedural form. We will annotate it with comments to try and communicate the concerns as we progress below. const myNumbers: Array<number> = [2, 4, 7, 13] let myNewNumbers: Array<number> = [] for (num in myNumbers) { let newNum = num + 5 myNewNumbers.push(newNum) } // myNewNumbers is now [7, 9, 12, 18]</code></pre> Looking at const myNumbers: Array<number> = [2, 4, 7, 13]</code> it is just providing the starting value for the process. It isn't controlling the flow of the process. Therefore, I would not classify it as part of the "control" concern. So let's tag it with an empty comment //</code>. const myNumbers: Array<number> = [2, 4, 7, 13] // other let myNewNumbers: Array<number> = [] for (num in myNumbers) { let newNum = num + 5 myNewNumbers.push(newNum) } // myNewNumbers is now [7, 9, 12, 18]</code></pre> If we look at the let myNewNumbers: Array<number> = []</code> on the first line of code and ask ourselves "Why is it there?" In this case, it is there to support the flow of this process by being the new array that the new values will live in. So let's flag it as "control" with a comment. const myNumbers: Array<number> = [2, 4, 7, 13] // let myNewNumbers: Array<number> = [] // control for (num in myNumbers) { let newNum = num + 5 myNewNumbers.push(newNum) } // myNewNumbers is now [7, 9, 12, 18]</code></pre> OK so what about the for (num in myNumbers) {</code>? Well, it is there to facilitate iterating through the collection. This sounds like controlling the flow of the process to me. So let's flag it as control. const myNumbers: Array<number> = [2, 4, 7, 13] // let myNewNumbers: Array<number> = [] // control for (num in myNumbers) { // control let newNum = num + 5 myNewNumbers.push(newNum) } // control // myNewNumbers is now [7, 9, 12, 18]</code></pre> OK, now we are onto let newNum = num + 5</code>. This is interesting because this is doing two things. The let newNum =</code> portion is controlling the flow of the process while the num + 5</code> portion is more about the work of computing a new value and not really about the flow control of the process. So let's flag it to specify this detail. const myNumbers: Array<number> = [2, 4, 7, 13] // let myNewNumbers: Array<number> = [] // control for (num in myNumbers) { // control let newNum = num + 5 // `let newNum =` control, `num + 5` some other concern myNewNumbers.push(newNum) } // control // myNewNumbers is now [7, 9, 12, 18]</code></pre> What about myNewNumbers.push(newNum)</code>? This seems to be code that manages how this process flows, so it seems to be control. Let's flag it as control. const myNumbers: Array<number> = [2, 4, 7, 13] // let myNewNumbers: Array<number> = [] // control for (num in myNumbers) { // control let newNum = num + 5 // `let newNum =` control, `num + 5` some other concern myNewNumbers.push(newNum) // control } // control // myNewNumbers is now [7, 9, 12, 18]</code></pre> At this point, we have classified all lines, statements, and expressions of code, and it feels like we have a pretty decent understanding of what code exists for the purpose of controlling the flow of the process and which code doesn't. In fact there are only two chunks of code that don't belong to the "control" concern, const myNumbers: Array<number> = [2, 4, 7, 13]</code> and num + 5</code>. Grouping & Separating Concerns</h3> Now that we have an understanding of at least some portion of the various concerns. We need to focus on conceptually grouping and separating them. In our case, we have the "control" concern and then two pieces of code that right now we know are not part of the "control" concern. The first step in separating concerns is to simply map those concerns to specific abstractions. To start let's take all the code that is "control" code and extract it to a function we will call foo</code> for now. Note: I use a garbage name for the function like foo</code> right now because I like to think about the concern conceptually after having moved the "control" code into it as I think it makes it easier to think about. If we do this we end up with something like the following. function foo() { let myNewNumbers: Array<number> = [] for (num in _________) { let newNum = _______ myNewNumbers.push(newNum) } } const myNumbers: Array<number> = [2, 4, 7, 13] num + 5 // myNewNumbers is now [7, 9, 12, 18]</code></pre> I put ________</code> in the function where the other concerns were interacting before. This helps us think about those elements in a more generic way. To make this function usable we have to fill in those ______</code> blanks. The only way to do that while maintaining the abstraction is to pass them in as parameters to the function. Our example would now look as follows. function foo(myNumbers: Array<number>, op: (number) => number) { let myNewNumbers: Array<number> = [] for (num in myNumbers) { let newNum = op(num) myNewNumbers.push(newNum) } return myNewNumbers } foo([2, 4, 7, 13], (num) => num + 5) // => [7, 9, 12, 18]</code></pre> By doing this we have not only made the foo</code> function usable. But we have made it more generic as we can now hand it any Array of numbers to operate on, and we can hand it any operation to perform on each element as long as it results in a number. At this point, we have successfully inverted control, and we have done so by identifying the control code, extracted it out into a function, and then using that function in its original place. It is important to understand that this is IoC because when we call the foo()</code> function. We are handing control over to the foo()</code> function for the lifetime of the foo()</code> process while we are still providing the details of the work it does. Generalize Types</h3> When using IoC it is common practice to generalize functions to remove unnecessary coupling to specific types from the "control" code. This is another mechanism for focusing only on the flow control of the process and less on the specifics of the types involved. This also has the benefit of generally making the extracted abstraction more flexible. If we start by looking at myNumbers: Array<number></code> and that it is only used in the for (num in myNumbers)</code> line. for (x in y)</code> doesn't care what type of array y</code> is. It just cares that it is an Array. If we make that change it would look as follows. function foo<A>(myNumbers: Array<A>, op: (number) => number) { let myNewNumbers: Array<number> = [] for (num in myNumbers) { let newNum = op(num) myNewNumbers.push(newNum) } return myNewNumbers } foo([2, 4, 7, 13], (num) => num + 5) // => [7, 9, 12, 18]</code></pre> This creates a temporary problem which is that the typing of op</code> would now also need to change so that it would take in a A</code>. And then we are immediately confronted with the next question. What type does op</code> return. From looking at the code, currently we have the values ending up in myNewNumbers</code> which is a Array<number></code>. So we would have to have it return a number</code> if we didn't change the code further. But it doesn't seem like there is any reason we shouldn't just allow that array to be generic as well so that op</code> can return a different type. This would allow the user to convert the types of elements within a collection if they wanted. We should also update the names of the parameters to represent more generic concepts. function foo<A, B>(collection: Array<A>, op: (A) => B) { let newCollection: Array<B> = [] for (item in collection) { let newItem = op(item) newCollection.push(newItem) } return newCollection } foo([2, 4, 7, 13], (num) => num + 5) // => [7, 9, 12, 18]</code></pre> The above foo()</code> function now supports our existing use case of wanting to add 5 to each element of the array but also supports a bunch of other cool things we can do because we have simply focused on separating an abstraction that focuses as much as possible on the control flow of the process. This is what Inversion of Control is all about. Naming Abstraction</h3> So now it seems we need to think about what our foo()</code> function facilitates as a process, and we should try and come up with a meaningful name. It lets us map each element of a collection from one value into another value. So maybe we should call it something like map()</code>. function map<A, B>(collection: Array<A>, op: (A) => B) { let newCollection: Array<B> = [] for (item in collection) { let newItem = op(item) newCollection.push(newItem) } return newCollection } map([2, 4, 7, 13], (num) => num + 5) // => [7, 9, 12, 18]</code></pre> This is actually how the standard map()</code> function is implemented. Conclusion</h2> We still have the "other" concern that we talked about before. What is interesting is that through this process that concern has also received its own abstraction, the op</code> closure. So now we can see fully how Inversion of Control is a specific type of Separation of Concerns, as the "control" concern and "map item value" concern have been separated by abstractions. We can also see how the standard map</code>, reduce</code>, and filter</code> functions are examples of IoC. But we have also seen what the process looks like for classifying and separating concerns looks like, which is a great skill that is applicable far beyond Inversion of Control. Making Code Reusable 2023-01-18T00:00:00+00:00 One thing that people don't seem to naturally understand is that to make code reusable you have to make it generic. To get a better understanding of this lets walk through some examples and see if we can get a better understanding of some good ways to make code more generic. Let's say we have a class called User</code> and user has a firstName</code> and lastName</code> properties. It might look something like the following. class User { constructor(public firstName: string, public lastName: string) { } }</code></pre> Now we find out that there is a need to access the user's fullName</code> within the application. Computed Property/Getter</h2> Most people's natural instinct is to simply add a fullName</code> getter to the User</code> class as a computed property. class User { constructor(public firstName: string, public lastName: string) { } get fullName(): string { return this.firstName + " " + this.lastName; } }</code></pre> This is great! Our immediate need is met. It lets us get the full name of a user as follows. const user = new User('Bob', 'Villa'); const fullUserName = user.fullName;</code></pre> However, this code isn't reusable. You need to ask yourself, "What does the action of combining a first name and last name conceptually have to do with a user?" The answer is, nothing other than that the user happens to have a first name and a last name. So how do we go about making this code more reusable? Well, step one is going back to the version of User</code> that didn't have a fullName</code> getter. class User { constructor(public firstName: string, public lastName: string) { } }</code></pre>Parameterized Function</h2> Let's try pulling this fullName</code> computed property out into a parameterized function and see how that feels. function fullName(firstName: string, lastName: string): string { return firstName + " " + lastName; }</code></pre> Now we can get the full name as follows. const user = new User('Alice', 'Jones'); const fullUserName = fullName(user.firstName, user.lastName);</code></pre> Ok now that we have extracted this into a function that is parameterized based on the firstName</code> and lastName</code> arguments it is able to combine any pair of firstName</code> and lastName</code> irrespective of where they came from. This is now reusable by definition. However, you may have noticed that the ergonomics is not quite as nice as it was with the Getter/Method approach. Let's see if we can improve the ergonomics a bit. Function Parameterized to Anonymous Interface</h2> One of the complexities we have is that with it strictly parameterized to firstName</code> and lastName</code> we have to create a mapping between the otherUser.firstName</code> and the otherUser.lastName</code> by passing them as parameters. It would be nice if we could simply pass a User</code> to the fullName</code> function. function fullName(user: User): string { return user.firstName + " " + user.lastName; }</code></pre> This simply moves the knowledge of how to access the firstName</code> and lastName</code> of the User</code> object from outside the function to the inside the function. This has a problem though. We just made the code no longer reusable, because it once again only works with objects of type User</code>. To address these we can use TypeScript's anonymous interface typing to specify that the function should accept any object that has both a property of firstName</code> and a property of lastName</code>. This looks as follows. function fullName(named: { firstName: string, lastName: string }): string { return named.firstName + " " + named.lastName; }</code></pre> Which allows us to now get the full name of the user as follows. const user = new User('Bob', 'Villa'); const fullUserName = fullName(user);</code></pre> Woot woot! We regained the generic and reusable nature that we had in the parameterized function but in the cases where we have an object that matches the properties of firstName</code> and lastName</code> it streamlines the usage. In the cases where the properties don't match that we simply have to do a mapping similar to the parameterized function example. Another cool thing about this is that it is a function that takes a single argument which now plays much nicer in a functional programming world. One question people generally ask at this point is where do I put this fullName</code> function though. If it isn't part of User</code> it shouldn't be in the same file as User</code>. I would argue you are correct. It has nothing to do with User</code>. This is where the concept of domain, a.k.a. problem space, a.k.a. concern comes into play. Organizing code based on concern is a much more useful strategy than by data as data is a byproduct of the concern. In this particular case we have a concern of "Name Composition" so I would move it to a TypeScript module named name_composition.ts</code> for the time being. You may have also noticed that we aren't 100% back to the ergonomics oh what we had at the very beginning with the Getter/Method approach. We can gain the full ergonomics back by adding the following getter to the User</code> class as follows. class User { constructor(public firstName: string, public lastName: string) { } get fullName(): string { return fullName(this); } }</code></pre> This gives us the best of both worlds as it gives us the reusability of the fullName()</code> function while also giving us the convenience of having the operation be discoverable via tooling on the User</code> class. In the end giving us the ability to get the fullName</code> with either of the two following approaches. const user = new User('Bob', 'Villa'); const fullUserName = fullName(user); const fullUserName2 = user.fullName;</code></pre> I personally prefer not having the getter on the User</code> class as it is just another layer of abstraction that isn't necessary. However, a lot of people value the discoverability of operations through tooling on the User</code> class to a high degree and think it is worth the cost. Function Parameterized to Named Interface</h2> Another option we have is to parameterize the function based on a named interface. For example, we could have an interfaced called Named</code> which would described our constraints of an object that has both a firstName</code> and lastName</code> property. interface Named { firstName: string; lastName: string; }</code></pre> This would then allow us to write our function as follows. function fullName(named: Named): string { return named.firstName + " " + named.lastName; }</code></pre> This is definitely more readable and easier to conceptually understand that the fullName()</code> function operates on conceptually any named object. However, it does add some more weight to the implementation. So I can see going back and forth on this. I think if there is more than one function that operates on the concept of a Named</code> object then it should definitely be formalized as a named interface. If it is a weird one off concept and function maybe it is fine to use the anonymous interface approach. Conclusion</h2> If we look back at what we did at a high level it looks as follows. question the belonging of some code</li> identify that it is not owned naturally by the class</li> extract it to a parameterized function</li> decide to parameterize with an interface (either anonymous/named) or leave it multiple parameters</li> </ul> This process is always the same. You just rinse and repeat the above steps over and over again as you are coding, and you will end up with a far more robust, meaningful, and reusable code base. Uptech Studio Continuous Integration Standards 2022-07-22T00:00:00+00:00 We are experimenting with a new variation of our workflow based on the Continous Integration Methology. It is designed to facilitate feature level engineering reviews while still enabling us to follow the Continuous Integration Methodology and operate purely on the main trunk. TL;DR</h2> Create Patch</h3> use Git Patch Stack</a> and follow Git Commit Standards</a> as you normal would.</li> </ul> Integrating a Patches</h3> integrate as soon as patch is complete and passes all automated checks verifying the requirements: buildable, testable, isolatable, releasble are met.</li> request review of the patch & immediately integrate the patchgps rr <patch-index> gps int <patch-index></code></pre></li> </ul> Note: The gps rr</code> is done solely to create notifications in GitHub as some people use those notifications to help with the "Keeping up with the code base". Keeping up with the code base</h3> gps fetch</code> or gps pull</code> show the upstream patches</li> scan patches to identify any that are of immediate interest related to what you are working on</li> patch created by someone you are mentoring, etc.</li> </ul> </li> review immediate interest patches by going to their GitHub Commit pages and commenting on the code with questions, suggestions, etc.</li> track patches you have looked at with Sublime Merge</a> commit selection state & auto refresh</li> read the commits to keep up with understanding how the code base is changing. To comment, ask questions, or make suggestions open the GitHub Commit page for that commit.</li> </ul> Note: Some people use the PR notifications in GitHub to help them do this. Ideally we would have some better tooling around this. Exploration in this is still happening. Complete Feature</h3> Once all your feature's patches are integrated: add a git ticket summary for each of the repositories involved as a comment, use git-ticket-summary</a></li> move the user story you have been working to the "Needs Review" column.</li> move onto your next task until someone reviews your feature</li> </ul> Feature Review</h3> You sould try and review a feature everytime you put a story in "Needs Review". pick a story in the "Needs Review" column that you want to review</li> drag the story you are going to review to "In Review"</li> review the feature's functionality in the dev environment to gain context and make sure it is meeting all the acceptance criteria</li> review the application architecture involved (how this feature works through the application architecture) while noting any logic/algorithms that need further focus</li> review logic/algorithm design in any areas of possible concern identified above</li> provide your feedback on the feature's application architecture & algorithms/logic in the ticket</li> if helpful you can comment on code in the commits, but make sure you still reference it in your comments in the ticket</li> if there are changes that need to be made move the ticket back to "In Progress" for the developer to address the concerns</li> rinse and repeat</li> once the story is good move the ticket to "Ready for Release"</li> </ul> References</h2> GitHub Commit Pages</h3> If you aren't familiar with GitHub's Commit pages. You can access them with the following url structure. https://github.com/uptech/git-ps-rs/commit/da5944f6157ff29cedaf8972e7f82f0d4e7a3a8c</code></pre> The SHA at the end of the URL, da5944f6157ff29cedaf8972e7f82f0d4e7a3a8c</code>, can be replaced with an SHA or short SHA of a commit get the URL for that commits GitHub Commit page. This is useful because GitHub allows you to inline comment on the within the commit similar to how you can within Pull Requests. Note: Some terminals support detecting and linking to the GitHub Commit Pages from the SHAs or short SHAs in the terminal buffers, e.g. Visual Studio Code, Alacritty</a>. Sublime Merge</h3> If you aren't familiar with Sublime Merge</a> it is a very nice cross-platform Git GUI that detects changes to the Git repo and auto refreshes state. This makes it extremely nice as you can be working in a terminal or another app and make changes and they automatically update and reflect in Sublime Merge</a>. In the context of this process I keep it up with the repository opened and as I gps pull</code> I can quickly and easily see the new patches in Sublime Merge</a> and checkout the code right there. If I find something I need to comment on I simply open that commit's GitHub Commit Page and start a discussion about the code. Journey to Small Pull Requests 2022-03-15T00:00:00+00:00 We were struggling to deal with our pull requests. We weren't getting quality peer reviews from our peers, we were having a hard time reverting commits when necessary, and had a hard time understanding our Git</a> history. So we decided to dig into this and see if we could get to the core of it and possibly come up with a solution. The first thing was just understanding all the of the issues we were running into and coming to the conclusion that they are all a side effect of big pull requests. Big pull requests effectively eliminate the value of peer review as the changesets are just too overwhelming for people to focus on.</li> silo changes from the team delaying feedback</li> silo changes from the team postponing integration</li> make conflict resolution more difficult</li> encourage poor git history & practices</li> make it hard to revert changes</li> result in a complicated git history making it hard to look back and understand how best to interact with or even understand changes</li> </ul> Why are they so big?</h2> After understanding the issues were coming from big pull requests we decided we should try and understand why our pull requests were so big and see if there was a good reason for it. We were following the GitHub Flow</a> and using feature branches, which in turn became our pull requests. Containing all the necessary changes for an entire feature turned out to be the primary reason that our pull requests were big. Not only that, we noticed that developers weren't caring about the individual commits at all and would have partial or broken commits in their branches or even arbitrary save point commits. This resulted in making reverting commits more difficult as well as breaking useful features of git like git bisect</code>. Beyond that it lended to the issues we had identified before. Alternatives</h2> So now we knew that if we wanted our pull requests to be small we couldn't use feature branches. Instead we would have to scope our changes by a smaller concept than a feature. To help us identify which scoping concepts made the most sense we researched many other development workflows. The two that stood out as being different were the Continous Integration Methodology and Trunk Based Development, as the others all seem to use feature branches. Both the Continous Integration Methodology and Trunk Based Development are focused around getting small, logical, buildable, testable, not-necessarily complete changes into mainline as quickly as possible. They both see this as a core principle to support other developers on the team being aware of the work being done and enabling contribution and planning around it. Digging further into how people follow these methodologies and scope their changes. It seems to range from a single small addition of an architectural concept (e.g. add method to class) to the addition of a small/initial architectural concept (e.g. adding a small class). One other difference is that people following these methodologies integrate code prior to it being utilized by other components. For example they would write a class and integrate that class prior to that class being used by whatever layers or features above it. Beyond that they even go as far as using Feature Toggles to facilitate including code in the code base but preventing it from being executed. This allows all of the code to be integrated and shared with the development team as quickly and early as possible while still preventing consumers from seeing incomplete features. What about Peer Review?</h2> One of our driving forces for looking at alternatives was to eliminate our problems with our peer reviews. Turns out that Continuous Integration & Trunk Based Development don't provide much guidance on that front. Many Continous Integration & Trunk Based Development teams forego what we think of as a peer review process coming from feature branches. Instead they opt for getting the small, logical, buildable, testable, but not-necessarily complete changes into mainline as quickly as possible and then in theory go back and review each of the commits after the fact. This is known as Post-commit review. What we classically think of as peer review with feature branches is known as Pre-commit review. The difference is simply Post-commit happens after a commit is integrated into mainline while the Pre-commit happens prior to a change being integrated into mainline. So we then started exploring the tools and processes around Post-commit peer review. Turns out that there isn't much there and the few tools that do exist are relatively antiquated and also require a full shift in terms of tooling and mentality. This was effectively a no-go for us as we work with many different clients and dev teams which all use GitHub</a>/Bitbucket</a> pull requests for peer review. So we needed to figure out a way to do small pull requests for Pre-commit peer reviews. Branches on, Branches, on Branches</h2> Following our only path forward for peer review, Pre-commit, we started making small pull requests scoped down just as people do in Continous Integration and Trunk Based Development. This was great as we were seeing the benefits of small pull requests and see that it definitely does address the issues we identified at the beginning. However there was a ton of extra overhead and complexities in terms of managing these branches locally. Generally we focus on implementing a specific feature/change. In order to do this we build up context around it in our mind so that we can plan and implement an appropriate change to achieve our goal. This is great when doing feature branches because there is a 1-to-1 mapping between that context and the branch. However, remember we are creating a number of pull requests that are small, logical, buildable, testable, and not-necessarily complete changes that when combined should accomplish the end goal of the feature/change. Continous Integration and Trunk Based Development teams sometimes refer to these smaller changes as Proof of Work. Just like when you were a kid in school and the teacher made you show your work on your math homework. Each of the small changes should be logical steps showing how you achieved your end goal. You might think it is natural to just create a branch for each of these logical changes. However, what you quickly find is that the logical changes begin to have dependencies on other logical changes you have made. This means for you to continue being able to develop the feature you effectively have to create your next branch on top of the previous branch and so on. You then end up with effectively a chain of branches composing up your feature. The only one you can actually open a pull request for is the inner most branch. This is because if you opened a pull request for any of the other branches it would also include the changes beneath it as well. Putting us back in the camp of big pull requests which we don't want to be in. So we continued our experiment and dealing with the branch management overhead by accepting that it is ok that the inner most branch is the only one you can open a pull request for until it is integrated into mainline. However we quickly ran into the next issue. What happens when someone provides feedback on the pull request of the inner most branch and you have to make changes to it. Well, you have to make changes to it and then effectively go rebase each of the branches above it ontop of eachother one by one all of over again. This made the branch management overhead just too much to deal with. Eliminate Branch Management Overhead</h2> So for small pull requests to be viable we determined we needed to find some way to eliminate or at least streamline all this branch management overhead. We pondered and struggled with this over time largely feeling like maybe it wasn't possible. However while researching development workflows of various teams we came across an old idea, the patch. Patches were how changes were managed and shared back in the day with many open source projects. A diff/patch would be created and emailed to the maintainer of the project. They would then review the patch and either apply it or reply back to the email requesting changes. Teams that use patches still generally require that they are small, logical, buildable, testable, and most importantly independent. Independent doesn't mean that one patch can't depend on another patch. But it means that if the patch doesn't have to depend on another patch it shouldn't. This is generally addressed by the need to be logical, but having the additional explicit requirement of independence is worth thinking about. In the cases where a patch must have a dependency on another patch it is simply handled by communicating the required order in which those particular patches must be integrated. Another interesting thing that maintainers do is locally manage a stack of patches they have recieved from contributors as they prepare changes to go into mainline. This got our brains rolling again and got us thinking maybe if we added this requirement of independence and we start conceptually thinking about our commits on main</code> as a stack of patches resting on origin/main</code>. It would eliminate the branch overhead all together. Not only that, if we used the pull.rebase = true</code> Git</a> feature it would force us to integrate our local stack of patches every time we pull. This would get us even closer to Continuous Integration and Trunk Based Development. So now we had all the branch overhead gone and we are closer to Continuous Integration and Trunk Based Development than we have ever been before. But we needed to be able to do the equivalent of sending a patch to a maintainer for review but using Pre-commit review via pull requests. Given that we are targeting independence and thinking about these commits as patches. It seemed natural when we wanted to request review of a patch to simply create a branch and cherry-pick the particular patch into that branch so that we could make a pull request from it. This worked and was far less overhead than all the branch management in the other path. Not only that it then allows Git</a> to help push us to independence by calling us out when we request review of a patch and it is not independent. Too Many Pull Requests?</h2> One concern we had initially with this workflow was how the increase in the number of pull requests would impact things both on the producing pull requests side as well as the reviewing pull reqeusts side. On the producing pull requests side we were pleasently surprised that we actually eliminated overhead. Before we used to have to write a pull request summary and description to give context about what the pull request is for and what it does in addition to all the commit messages. This went away because when you have a single commit branch for a pull request it grabs the commit summary and message as the pull request summary and description. You could argue that there must be additional overhead in terms of the context that must be included in the commit messages. For us there wasn't as our Git Commit Standards</a> already required us to write good commit messages that include the necessary context. If your team is coming from a place of not having good commit messages there will likely be a bit of overhead in terms of commit message writing. It is well worth it though and will pay dividends not only in supporting this workflow but also in supporting Git</a> history spelunking. We were also pleasently suprised when we realized that we stopped caring about pull requests being reviewed immediately. It turns out that this is a wonderful side effect of having your patches in a stack. You can continue working on the feature without any branch overhead irrespective of it takes 30 mins or 2 days to get a review. On the reviewing pull requests side of things the act of reviewing pull requests becomes a lot easier as everything you are reviewing is logically scoped, buildable, testable, etc. It allows you to really focus in on the change and how it plays out with the application architecture. It also makes it easier to allow your self to do some deep focus work as your team doesn't have the incessant need to ping you about their pull request to get it reviewed instantly. There are definitely more pull requests. But when you come up for air from your deep focus you just tackle reviewing a few pull requests instead of one giant one. We have found that the organization and separation of these pull requests has only increased the speed and quality of our peer reviews. It is probably also worth mentioning that we use a product of ours, Pullwalla</a>, to identify which pull requests need review. This also helps us stay in deep focus. Outside-In & Inside-Out</h2> In addition we were unsure exactly how things would pan out in situations where we have to go down a path to figure out how to implement something. Turns out that following best practices and doing outside-in development works great with this workflow. The key addition is that once we get to the inner most dependency we work our way inside-out requesting review of the patches. Basically what ends up happening is that you create a bunch of logical WIP commits as you work from the outside-in. You do this until you get to the the inner most dependency (C) that has all the dependencies it needs. That patch (C) of the inner most dependency is what is fully baked and ready to have a pull request open. You are confident that your changes in C met the needs of B because you just walked through from the outside-in understanding the needs before you implemented C. A - WIP: Add ability for user to Login B - WIP: Add login ability to authentication service C - Add login() function to core underlying authentication library</code></pre> Then you step back up to then next level above that (B) and you refine that patch updating it to use the stuff provided by C and make it no longer a WIP commit. A - WIP: Add ability for user to Login B - Add login ability to authentication service C - Add login() function to core underlying authentication library</code></pre> At this point you open a pull request for B. Then you start working on refining patch A, using the bits that you implemented in B and get it to a good place. A - Add ability for user to Login B - Add login ability to authentication service C - Add login() function to core underlying authentication library</code></pre> At this point you have a patch that technically implements the feature from a user perspective & adds the associated e2e tests. This is still a small, logical, buildable, testable, patch that you open a pull request for. The peer reviewer understand that this patch adds the feature and therefore when they review this pull request they do the full feature review as well as code review. Generally these inner patches are paired with unit tests while the outer most feature patch is paired with e2e tests. There are many ways this can vary. For example with someone not doing outside-in development or maybe you have multiple internal dependency patches that aren’t intertwined. I think the important thing is just that the pull requests are opened from the inside-out. Also one of the beautiful things about this workflow is that it decouples you from caring about how quickly your pull requests get reviewed because you are able to still continue development of your feature irrespective of how long it takes someone to review your pull requests. This happens thanks to the fact that the patch stack continues to hold the patches even after requesting review. Further details on this process of outside-in and inside-out in relation to this workflow are covered in our article How we should be using Git</a>. Ticketing Alignment</h2> When we were doing feature branches before the ticketing alignment was trivial. We basically had a 1-to-1 mapping between a user story and feature pull request. With the Git Patch Stack methodology we had to make a choice. Do we have multiple patches reference a single user story or do we modify how we do ticketing so that it better aligns with the logical patches. After thinking about it briefly we decided that it makes the most sense to just have multiple patches reference the same user story. Each of the patches could also reference any related sub-tasks that belong to the parenting user story. The key for this to be successful is really just providing the necessary context in the patch commit messages. The alternate path of changing how we define and structure tickets seemed like a bad idea as it would have required not just engineering to change but also the product and project managers. Git Patch Stack</h2> After using this workflow and seeing and feeling the value that small pull requests provide. We decided we could relatively easily streamline the little bit of branch overhead involved with this workflow and further enforce the conceptual thinking around patches by creating a simple Git</a> extension. So we created Git Patch Stack</a> and it has only further solidified the workflow, the mentality around it, and the vast benefits that fall out of small pull requests and this workflow in general. For a deeper dive on Git Patch Stack</a>, this workflow and concepts to help identify these smaller logical chunks of work, see our post on How we should be using Git</a>. Uptech Studio Pull Request Standards 2022-03-01T00:00:00+00:00 We require that Pull Requests have the following characteristics. small</li> logical</li> buildable</li> testable</li> releasable</li> </ul> Small</h3> It is crucial that pull requests are small. The quality of peer review exponentially decreases as the size of the pull request increases. If you are going to have PRs in the hundreds or thousands of lines of change range you may as well just not do peer review. Here at Uptech Studio</a> we believe in producing the highest quality code that we can and therefore we must have small pull requests. If you are interested in how we got to small pull requests, checkout our post Journey to Small Pull Requests</a>. Logical</h3> It is also important that pull requests are logical. This means that the bounds of the content are well ordered and have clear & sound reasoning. You might say well a feature is logical. Well, you would be correct. However, having a pull request contain an entire feature is in contradiction with the requirement of a pull request being small. Therefore we need to find some other logical bounding for the change. Generally application architecture concepts are a great aid for this. For example you could have a pull request that adds a method to a service to allow consumers to get a collection of items which would be logically bound around the change to that service. It could also be that you are making a cross-cutting change that is still logical. For example you could have a pull lequest that changes code from using strings to symbols which is spread across multiple application architecture concepts but is still logical. Having a pull request be logical is crucial not only for understanding the evolution of the code base over time but is crucial to support functionality in Git</a> like bisect</code> and revert</code>. Buildable</h2> Pull requests also must be buildable. Meaning that the state of the code in the pull request has to be in such a place that the library/framework/application/suite can be built without error. This is crucial again to support functionality in Git</a> like bisect</code> and revert</code> but is also import to not impede the development process of the team. Testable</h2> A pull request being testable means that the change has automated tests covering it and that the test suite is able to be run with those tests included. In cases where code isn't able to be tested with automated tests, maybe some UI change, then the changes have to be manually testable and the pull request description should include enough information to educate someone on how to manually test it. This is crucial, especially when making smaller pull requests because it doesn't make sense to pull down a pull request and manually test changes to an internal application architecture concept. So instead of getting our confidence from manual testing we get it through making sure that we have well written automated test coverage and that they are passing. Releasable</h2> We also require a pull request to be releasable. This doesn't mean that you would idealisticly want to cut a release with just this one pull request. But it means that if push came to shove and you needed to for some reason the application/library would be in such a state where it could be released. This generally is a pretty easy requirement to meet as you can have code in your code base that is buildable & testable that isn't used, and that is totally fine. The complexity with this requirement comes when you are dealing with changes that are visible to the consumer of the library/application. That is where you have to then ask yourself if I make this change to the UI if push came to shove would this change be releasable? Remember it isn't that if it was released it would be the ideal release. It is that it would be possible to release it. Backend Web Framework Design Exploration - Part 3 2022-01-05T00:00:00+00:00 This is the third in a series of articles I am writing in which I do a Design Exploration of a Backend Web Framework with the following constraints. only use functions & data - no classes w/ methods</li> use strong typing (via TypeScript) - no use of any</code></li> trivially testable via basic unit style state based tests</li> </ul> If you didn't start at the beginning I highly recommend it, Backend Web Framework Design Exploration - Part 1</a>. Note: Within this article I will include simplified code examples. If however you are interested in the real code & tests I used to do this design exploration you can see them via the commits at my be-wf-design-exploration</a> Git repository. Dependencies Constructed per Request</h2> Picking up where I left off in the previous article. I need to figure out how this web framework should support Dependencies that need to be constructed fresh per Request. This is something that most web frameworks support via their dependency injection frameworks. Of course I have to manage this while following the 3 key constraints above. Previously I had the fooHandler</code> with a dependency of dbPool</code>. export function fooHandler( dbPool: DatabasePool ): wf.RouteHandler { return function (request: wf.Request): wf.Response { return { status: 200, body: dbPool.id }; }; }</code></pre> I am going to expand on this example with the DatabasePool</code> by saying that our fooHandler</code> also needs another database pool. However, this one needed to be constructed fresh with each request. Dependency Injection Frameworks</h3> Looking at all the dependency injection frameworks I have worked with. They are effectively a set of functions that are registered often by type or name. Then at some later point in time you call a dependency injection framework function to instantiate an instance by either a type or name. Often times the registered function is executed only the first time and then memoized as a singleton. Effectively making them global singletons. Functions as Parameters</h3> Thinking about dependency injection frameworks and the fact that I need something that can at some later point in time give me back in instance of this other DatabasePool</code>. It sounds very much to me like a function that is simply responsible for knowing how to construct this other DatabasePool</code> is the key. In fact it can just be an explicit dependency of the fooHandler</code>. This aligns at a high level with how common Dependency Injection frameworks work to some extent while also abiding by the constraints. Specifically the explicit dependency declaration constaint. Which in turn supports the trivial testing requirement. Applying this concept to a handler might look something like this. export function fooHandler( dbPool: DatabasePool, getOtherDbPool: () => DatabasePool ): wf.RouteHandler { return function (request: wf.Request): wf.Response { const otherDbPool = getOtherDbPool(); return { status: 200, body: `${dbPool.id} & ${otherDbPool.id}` }; }; }</code></pre> With the usage in the routes builder looking as follows: export function buildRoutes(dbPool: DatabasePool, getOtherDbPool: () => DatabasePool): wf.Routes { return [ { pattern: "/foo", handler: fooHandler(dbPool, getOtherDbPool) }, { pattern: "/bar", handler: barHandler() }, { pattern: "/health", handler: healthHandler() }, ]; }</code></pre>Maybe use Environment</h3> Some frameworks I have worked with use a concept known as Environments for dependency injection. Basically, it is just a formalized Type that you explicitly pass in as a parameter to your function that houses all your dependency injection dependencies. The environment for fooHandler</code> might look something like the following. class FooEnvironment { dbPool: DatabasePool; getOtherDbPool: () => DatabasePool; }</code></pre> Then I could use it in place of the parameters of fooHandler</code>. export function fooHandler(environment: FooEnvironment): wf.RouteHandler { return function (request: wf.Request): wf.Response { const otherDbPool = environment.getOtherDbPool(); return { status: 200, body: `${environment.dbPool.id} & ${otherDbPool.id}` }; }; }</code></pre> Looking at this I think the real value is that it facilitates grouping of dependencies together. I think this would be valuable for grouping dependencies maybe at a module level. I guess the only other value I would see from this on a handler basis is just to provide a more defined convention for how a user would want get dependencies into the handler. However, I could also just provide examples probably without the need for this. Review & Next Up</h2> Looking at all of the above. The passing a function as an explicit dependency to get get an instance of a something per request seems like the most direct solution. It also meets all the constraints. The Environment concept might make more sense if later on I decided that requiring some modularization makes sense and therefore grouping dependencies by module makes sense. But until then I think starting with the function paramater technique makes the most sense. Both of these techniques enable more precision in terms of instance creation than being bound to a Request. This is because the handler can call the function multiple times. It is worth recognizing that it is really the builder function technique that is still facilitating all of this. It is just that being able to pass a function as a parameter combined with that approach facilitates this specific use case. This concept of builder function is used generally to implement the concept of currying</a> in function programming. Next up is figuring out midddleware & guards and how they might work in this framework. Backend Web Framework Design Exploration - Part 2 2022-01-03T00:00:00+00:00 This is the second in a series of articles I am writing in which I do a Design Exploration of a Backend Web Framework with the following constraints. only use functions & data - no classes w/ methods</li> use strong typing (via TypeScript) - no use of any</code></li> trivially testable via basic unit style state based tests</li> </ul> If you didn't start at the beginning I highly recommend it, Backend Web Framework Design Exploration - Part 1</a>. Note: Within this article I will include simplified code examples. If however you are interested in the real code & tests I used to do this design exploration you can see them via the commits at my be-wf-design-exploration</a> Git repository. Dependency Injection for Handlers</h2> Picking up where I left off in the previous article. I need to figure out how this web framework should support injecting dependencies into handlers while still supporting the 3 constraints above. A very common use case for this would be for a handler to need to interact with a database. This is generally done through a database connection pool. So I am going to use that as an example use case. Builder Function Technique</h3> In the previous article I used the builder function technique while implementing the router</code> function so that I could give that scope dependencies while keeping the router signature the same (Request) => Route</code>. I can simply use the same technique here but applied to a handler if it needs a dependency or two like a dbPool</code> or something. function fooHandler(dbPool: DatabasePool): RouteHandler { return (request: Request): Response => { let dbConn = dbPool.getConnect(); let queryResults = dbConn.query("some query"); ... return new Response(...); }; }</code></pre> This changes how I have to use it in the route definitions as we have to pass the dbPool</code> into the fooHandler</code> builder function as follows. const routes = [ { pattern: "/foo", handler: fooHandler(dbPool) }, { pattern: "/bar", handler: barHandler }];</code></pre> This is functionally just fine. The handler resulting from the fooHandler</code> builder function is trivial to test via state based unit testing. However, if I try to do an end-to-end test lets say as follows. test("end-to-end request -> response", () => { const request: wf.Request = { path: "some/path" }; const response = process(request, router(routes, notFoundRoute)); expect(response.status).toBe(200) });</code></pre> I would have a big problem because it would be using the real database pool. This is something that I want to be able to stub out in our end-to-end unit tests. This builder function technique can come in and save the day again. If I apply the concept to building the routes and then use explicit dependency declaration I get something like the following. function buildRoutes(dbPool: DatabasePool): Routes { return [ { pattern: "/foo", handler: fooHandler(dbPool) }, { pattern: "/bar", handler: barHandler }]; }</code></pre> In this case I am not building another function as I have in the past, but instead I am building a collection of routes. Which now enables the simple unit style testing at the higher end-to-end level while still being able to stub out the dbPool</code>. test("end-to-end request -> response", () => { const request: wf.Request = { path: "some/path" }; const response = process(request, router(buildRoutes(dbPool), notFoundRoute)); expect(response.status).toBe(200) });</code></pre> The buildRoutes</code> builder function can be used to support other dependencies as well. Review & Next Up</h2> Looking at things thus far I think design wise it is still in a very good place. This idea of using a builder function to build the routes helps me meet all the constraints. It also provides a natural logical grouping of routes. For example I could do something like define a builder specific to a particular REST resource and compose its routes together with those of another builder. const routes = Array().concat( buildResourceARoutes(dbConn), buildResourceBRoutes(dbConn)) const response = process(request, router(routes, notFoundRoute));</code></pre> Beyond that it can provide a natural grouping mechanism for dependencies. const routes = Array().concat( buildRoutesNeedingDb(dbConn), buildRoutesNeeding(depA, depB, depC)) const response = process(request, router(routes, notFoundRoute));</code></pre> The above two concat</code> examples don't read the nicest. But that is easy enough to fix with the addition of another function to make composition of routes easier if later on I decide it is worth it. Next up I am planning on tackeling how I address dependencies that should be constructed fresh for each request. Backend Web Framework Design Exploration - Part 1 2021-12-30T00:00:00+00:00 This is the first in a series of articles I am writing in which I do a Design Exploration of a Backend Web Framework with the following constraints. only use functions & data - no classes w/ methods</li> use strong typing (via TypeScript) - no use of any</code></li> trivially testable via basic unit style state based tests</li> </ul> Note: Within this article I will include simplified code examples. If however you are interested in the real code & tests I used to do this design exploration you can see them via the commits at my be-wf-design-exploration</a> Git repository. Why the constraints?</h2> It has been my experience that applying constraints like this pushes you to think about problem solving in different ways. My hope is that by forcing myself to think within these constraints that I will have some valuable take aways in terms of deeper understandings of design principles and best practices. Why these constraints?</h2> Throughout my career I have used a lot of different backend web frameworks in a lot of different languages. Sadly I can't think of one that actually facilitates end-to-end request -> response</code> testing in a simple unit testing style. Generally you have to do a bunch of horrible magic to facilitate dealing with dependencies (database, etc.) or use a bunch of unique magic provided by the framework. I believe this magic is unnecessary nonsense and that it is possible to design a framework in such a way that would naturally facilitate simple unit style testing. Hence, the constraint of requiring everything be trivially testable via basic unit style state based tests. Additionally, I am constantly faced with developers having this strange impulsive fear of functions. These developers generally build classes that have no real state and no need to actually be a class. In fact they seem to simply use the class to implicitly share dependencies. I think this conciously or unconciously boils down to people thinking that explicit dependency declarations via function paramaters are bad. I believe this is due to people worrying about "threading" dependencies through layers and layers of functions and not understanding how modularity works. My instinct is that explicit dependency declaration is extremely valuable. Not only does it facilitate testing but it can also act as an indicator of poor design. For example seeing heavy parameter "threading" should tell you something is likely off with the design. Therefore I have included the constraint of only using functions & data to try and prove not only to myself but hopefully to others that using explicit dependency declarations is extremely valuable. Beyond that, over the years I have moved further and further away from dynamic languages and closer to static strongly typed languages. The amount of aid the the compiler provides these days is amazingly valuable and I believe it outways the limitations. So I thought it would be interesting to see what sort of implications there are from having that requirement in this context. Guiding Architectural Concept</h2> For the sake of discussion lets say we want a web framework that would help us build a REST API. REST APIs are built on top of the HTTP protocol and the HTTP protocol is a Request-Response protocol. That means that a client makes a request with some data and the server responds with some data. When I think about this Request-Response concept it feels very similar to the concept of a function. function add(x: number, y: number): number { return x + y; }</code></pre> When you call the function add(5, 2)</code> it is equivalent in my eyes to a client making a request to a server with the data { "x": 5, "y": 2 }</code> and getting back a response, in this case the return value of the function. Given such similarity to a concept that exists naturally in all these languages I am going to think of each request-response pair as a function in the hope that it can help guide my design. What a Backend Web Framework does?</h2> I am going to start by focusing on the core functionality of the framework and see if I can evolve it from there. At the most basic level I think a Backend Web Framework needs to do the following. Receive a request</li> Identify the request & it's associated handler</li> Execute the matched handler</li> Receive the response from the handler</li> Transport the response back to the client</li> </ol> Design 001</h3> Naively this might look something like the following. However this design has one major problem. The router</code> doesn't meet single responsibility principle</a>. Infact in this design it is responsible for doing the following. mapping a request to a handler</li> coordinating execution of the handler & tunneling the response</li> </ul> This problem would prevent the router</code> from being tested in isolation from the execution of the mapped handler which goes against defined constraints. Design 002</h3> Taking the learnings from the first design. I decided to separate the concerns of the mapping a request to a handler and the coordination of the execution of a handler. This shift now makes it so the process request & router abide by the single responsibility principle</a>. In turn enabling testing of the router configuration in isolation. Initial Framework</h2> The following is the initial framework implementation based on Design 002. To see the full implementation at this stage checkout commit 9bd3e3</a>. export interface Request { path: string; // ... } export interface Response { status: number; // ... } export type RouteHandler = (request: Request) => Response; export interface Route { pattern: string; handler: RouteHandler; } export type Routes = Array<Route>; export type Router = (request: Request) => Route; function routeMatches(request: Request, route: Route): Bool { return true; } export function router(routes: Routes, notFoundRoute: Route): Router { return function (request: Request): Route { const matchedRoute = routes.find((route) => routeMatches(request, route)); if (matchedRoute) { return matchedRoute; } else { return notFoundRoute; } }; } export function process(request: Request, router: Router): Response { return router(request).handler(request); }</code></pre> The above seems pretty reasonable at least at a high level. Probably the most interesting technique I have used thus far is in the implementation of the router</code> function. It is simply a builder function that builds the actual instance of type Router</code> and returns it. It accomplishes this by returning an anonymous function that matches the Router</code> type signature. That by itself isn't really that interesting. However the fact that the builder function can take in parameters which are then available to the scope of actual router implementation (anonymous function) is extremely valuable. It is what enables us to feed dependencies into a function while having it's signature map to typed signature that knows nothing about those dependencies. This technique is commonly used in functional programming. To make sure it supports all the levels of testing I want before adding support for anything else. I whip up a couple tests. test("routing configuration", () => { const request = ...; const route = router(routes, notFoundRoute)(request) expect(route).toStrictlyEqual({ pattern: "some/path", handler: somePathHandler }); }); test("end-to-end request -> response", () => { const request: wf.Request = { path: "some/path" }; const response = process(request, router(routes, notFoundRoute)); expect(response.status).toBe(200) });</code></pre> The above allows us to test the following things. end-to-end test with a request</li> test routing configuration in isolation</li> test the route handler in isolation</li> </ul> Usage</h2> To get a better feel for what this framework might actually look like to use in this state. I have broken down a rough example. import * as wf from "./wf"; function fooHandler(request: wf.Request): wf.Response { return { status: 200 }; } function barHandler(request: wf.Request): wf.Response { return { status: 200 }; } function notFoundHandler(request: wf.Request): wf.Response { return { status: 404 }; } const routes = [ { pattern: "/foo", handler: fooHandler }, { pattern: "/bar", handler: barHandler }]; const router = wf.router(routes, notFoundRoute); // Normally this request would come in via an HTTP Server and get translated // into a wf.Request object and in turn handed to the wf.process() function. // However, this works as a means of simulating this for the development of // the framework without needing to build the HTTP Server portion. const request: wf.Request = { path: "/foo" }; const response = wf.process(request, router);</code></pre> I threw the above into a single file to facilitate seeing it all at once. In reality this would obviously be broken up into different files. Review & Next Up</h2> Looking at what I have thus far I am feeling pretty good about the design. It feels simple and clean yet still extremely robust and flexible while meeting all of the outlined constraints. Next up is going to be interesting as I need to figure out how to ideally facilitate dependency injection within the framework. Git Patch Stack - Add a Patch 2021-09-23T00:00:00+00:00 </iframe> </div> Today we are focused on Git Patch Stack</a>, an open source tool we developed and use for a patch stack based workflow in git</code>, instead of doing feature branches. What is Git Patch Stack?</h2> You might still be asking, "Well, what is that?" Basically it's a workflow that we can use to conceptually create logically independent patches, and we have a stack of them. So think of a stack of plates. Except, you can do things like reorder them, squash them together, split them apart, etc. In this workflow you generally work all on the main</code> branch, and it facilitates you building on top of the changes you have already made. Rather than you having to worry about creating a branch that is based on top of another branch and then based on top of another branch and so on and so forth. It also promotes proper git</code> usage so that you have logical chunks. Enabling tooling like git bisect</code> to actually work. To learn more about this workflow checkout my post, How we should be using Git</a>. Adding Patches to your Stack</h2> Today we're going to focus on how we create patches on our stack and what is a stack in terms of git</code>? Git Patch Stack</a> is really just a layer of tooling built directly on top of git</code> itself. So if we look at our git</code> tree here, we have a small project. It has an initial skeleton of a Rust project. That's just a Hello World program right now, and we can see we have a branch called main</code> that were checked out on. That's why HEAD</code> points to it, and we then have an upstream main</code> with a remote of origin</code>. That happens to be where main</code> points to on our GitHub repository for this. Technically, in Git Patch Stack</a> terms, any branch that has an upstream is a Patch Stack. So you can have as many of these Patch Stacks as you want, as long as they are branches that have remote upstreams. To add patches on to a stack, you first have to be checked out on a branch that has an associated upstream. In our case, main</code>. Make a change</h3> Then we just make changes like we normally would in git</code>. We add a commit and that commit conceptually becomes a patch. So let's do that real quick. Let's go look at our source code here. We have a main()</code> function. fn main() { println!("Hello, world!"); }</code></pre> Let's say that we want to add another function called foo()</code> for some reason, and we want this function to print "Foo", nothing too crazy. So we add a function. We're not even going to use the function. We're just going to add it to the code base so that we can use it in the future. fn main() { println!("Hello, world!"); } fn foo() { println!("Foo"); }</code></pre> That's it. Now we just do a git diff</code> to verify our changes. diff --git a/src/main.rs b/src/main.rs index e7a11a9..0e47771 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,3 +1,7 @@ fn main() { println!("Hello, world!"); } + +fn foo() { + println!("Foo"); +}</code></pre> I have a git alias of di</code> setup so I can just run git di</code> to get the diff quicker. Git Aliases</a> </blockquote> Once we verify the change is what we want, then we stage that change. We can stage that change by doing a git add</code> of that file. So far, we're just using all normal git</code> commands, and now it's staged. So we can do a git diff --cached</code> and that will show us all the changes that are staged as a diff. diff --git a/src/main.rs b/src/main.rs index e7a11a9..0e47771 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,3 +1,7 @@ fn main() { println!("Hello, world!"); } + +fn foo() { + println!("Foo"); +}</code></pre> It looks good. Looks like the changes we want. I have a git alias of dc</code> setup so I can just run git dc</code> to get the diff quicker. Git Aliases</a> </blockquote> Add Patch to Stack</h3> If we do a git commit</code>, it pops up my editor and I can enter the commit message Add foo() function So that in the future we can print the message when necessary.</code></pre> Ideally, we provide more context but because this is a contrived example. We don't really have it. Now we have a commit. If we look at our git</code> tree, you can see we have a commit here that we're pointing to on main</code>, which is the "Add foo() function" patch. And then underneath that in the tree, we have the initial skeleton commit we had before. Now, if we run git ps ls</code>, which is how we list our stack of patches. We can now see that our stack of patches consist of one commit, the commit that we just added on there, aka a patch. Add another Patch</h3> So let's add another one real quick just so we can see what it's like to have two. Let's just copy this and paste that, and change this word to bar, change this to bar and then we'll add another function. fn main() { println!("Hello, world!"); } fn foo() { println!("Foo"); } fn bar() { println!("Bar"); }</code></pre> Now we just git add</code>, git dc</code> to verify our staged code. All looks good. We commit it with the message. Add bar() function So that we can print the "Bar" message in the future.</code></pre> Right now we should have two patches in our stack. We have patches zero, which is the index of that patch currently. And we have patch index one. The short sha of those commits, a.k.a. patches visible to the right of the shas. We can also see the patch summaries. As you get into Git Patch Stack</a> further, you'll see indications here that will indicate things like whether that patch has been requested for review, the patch has been published, or there's been changes since you requested review. But for now that's it. That's how you add a patch to the top of your stack. Hope you enjoyed. SwiftUI: Composable Button Styles 2021-06-14T00:00:00+00:00 One frustrating/annoying things about SwiftUI Buttons for me has been the mechanism for styling them. In order to get information about the state of the button (e.g. if it is pressed, not pressed) you have to build a struct</code> that implements the ButtonStyle</code> protocol. This would generally look something like the following. struct MyColoredButtonStyle: ButtonStyle { let pressedBackgroundColor: Color = ... let notPressedBackgroundColor: Color = ... func makeBody(configuration: Configuration) -> some View { return configuration.label.background(configuration.isPressed ? pressedBackgroundColor : notPressedBackgroundColor) } }</code></pre> Then to use it you simply do the following: Button(...).buttonStyle(MyColoredButtonStyle())</code></pre>The Problem</h2> This is great and all until you decide that you also want your button to have an animated scale effect tied to the isPressed</code> state as well. You might think you could just create a new MyAnimatedScaleButtonStyle</code> struct and tack it on like so. Button(...) .buttonStyle(MyColoredButtonStyle()) .buttonStyle(MyAnimatedScaleButtonStyle())</code></pre> Sadly SwiftUI only takes the first button style in the chain and applies it. The rest are ignored. This means you basically have to create a super unique button style for each particular use case and have all your styling slammed into it. A Solution (A Micro Framework)</h2> Taking from Functional Programming we can effectively use Function Composition to facilitate making a tiny lightweight framework to facilitate building composable button styles. typealias ButtonStyleClosure<A: View, B: View> = (ButtonStyleConfiguration, A) -> B precedencegroup ForwardComposition { associativity: left } infix operator >>>: ForwardComposition func >>> <A: View, B: View, C: View>( _ f: @escaping ButtonStyleClosure<A, B>, _ g: @escaping ButtonStyleClosure<B, C> ) -> ButtonStyleClosure<A, C> { return { configuration, a in g(configuration, f(configuration, a)) } } struct ComposableButtonStyle<B: View>: ButtonStyle { let buttonStyleClosure: ButtonStyleClosure<ButtonStyleConfiguration.Label, B> init(_ buttonStyleClosure: @escaping ButtonStyleClosure<ButtonStyleConfiguration.Label, B>) { self.buttonStyleClosure = buttonStyleClosure } func makeBody(configuration: Configuration) -> some View { return buttonStyleClosure(configuration, configuration.label) } } extension Button { func composableStyle<B: View>(_ buttonStyleClosure: @escaping ButtonStyleClosure<ButtonStyleConfiguration.Label, B>) -> some View { return self.buttonStyle(ComposableButtonStyle(buttonStyleClosure)) } }</code></pre> This allows us to define custom ButtonStyleClosure</code> functions. The follow are just a few examples. But the idea is that you can create whatever of these ButtonStyleClosure</code>s you want and then use Function Composition to combine them. struct ButtonStateColors { let pressed: Color let notPressed: Color } func scaledButtonStyle<A: View>(_ configuration: ButtonStyleConfiguration, _ view: A) -> some View { return view .scaleEffect(configuration.isPressed ? 0.98 : 1) .animation(.spring(response: 0.2, dampingFraction: 0.5, blendDuration: 0.5)) } func roundedButtonStyle<A: View>(_ configuration: ButtonStyleConfiguration, _ view: A) -> some View { return view.cornerRadius(8) } func defaultPaddingButtonStyle<A: View>(_ configuration: ButtonStyleConfiguration, _ view: A) -> some View { return view.padding() } func coloredButtonStyle<A: View>(_ configuration: ButtonStyleConfiguration, _ view: A) -> some View { let backgroundColors = ButtonStateColors( pressed: Color("cta_button_highlight_color"), notPressed: Color("cta_button_color")) let foregroundColors = ButtonStateColors( pressed: Color("cta_button_text_color"), notPressed: Color("cta_button_text_color")) return view .background(configuration.isPressed ? backgroundColors.pressed : backgroundColors.notPressed) .foregroundColor(configuration.isPressed ? foregroundColors.pressed : foregroundColors.notPressed) }</code></pre> Allowing us to simply use these with our Function Composition operator we defined in the framework as follows: Button(...) .composableButtonStyle(defaultPaddingButtonStyle >>> coloredButtonStyle >>> roundedButtonStyle >>> scaledButtonStyle)</code></pre>Conclusion</h2> This gives us lots of reuse and facilitates making more generic button styles while allowing us to use them throughout our app rather than having to make unique button styles for each case. It does have a limitation though. It would be ideal if we could create higher level functions by nesting closures. Which would give us even more power. For example we could create a more general foregroundColorButtonStyle(colors:)</code> function like the following. func foregroundColorButtonStyle<A: View>(_ colors: ButtonStateColors) -> (ButtonStyleConfiguration, A) -> some View { return { configuration, view in view.foregroundColor(configuration.isPressed ? colors.pressed : colors.notPressed) } }</code></pre> In theory this would enable us to be able to be even more composible. However, it turns out Swift 5.4.1 doesn't support some View</code> in closures. So the above sadly won't work. I briefly tried to come up with a nice workaround but wasn't able to get anything nice to work. Despite this one limitation it is still far superior in my mind compared to what SwiftUI provides out of the box. Changelog Standards 2021-05-01T00:00:00+00:00 At Uptech Studio</a> we keep a CHANGELOG.md</code> following the keepachangelog.com</a> standard with all libraries and applications we create. We do this by using the git-cl</a> tool that we built. It enables us to get all the values of keeping a CHANGELOG.md</code> while not having to deal with the workflow impedances. To understand more about git-cl</a> you can checkout our Keep a Changelog without Conflicts</a> blog post which also includes a video introducing the tool. Codebase Organization 2021-05-01T00:00:00+00:00 TLDR</h2> The following is the TLDR of how I believe you should organize your code base. organize code according to architecture not types of architecture element types</li> if in framework that does organize by architecture elment types see if you can organize by architecture and then move the organization by element types into the folders representing the architectural components.</li> if you can't, follow the framework convention up until you reach natural boundary (e.g. lib</code> folder), then switch to organizing by architecture</li> </ul> </li> </ul> Major Strategies</h2> There are basically two major strategies people use to organize their codebases. around types of architecture components (e.g. models</code>, views</code>, controllers</code>)</li> around actual application architecture</li> </ul> Around Types of Architecture Components</h3> Frameworks like Rails and NestJS are examples that follow this strategy where they have a folder for models</code>, controllers</code>, etc. Organizing your codebase around these types of architecture components has the following advantages. you don't have to think very hard when you are trying to figure out where to put something. Controllers go in the controllers</code> folder, models</code> in the models folder, etc.</li> </ul> This benefit however is not always true because there is generally a boundary where this falls apart as the framework doesn't provide direction. The lib</code> folder is often one of these boundaries. The framework provides guidence in terms of organization right up until that point and then all of the sudden lib</code> is just a dumpster full of stuff. Organizing your codebase around these types of architecture components also doesn't help people understand anything from the codebase other than that there are things called controllers</code>, models</code>, and views</code>. But it doesn't give you any knowledge in terms of how certain controllers, models, or views relate to one another. You have no idea from the organization of the codebase how to find a particular part of the application. Around Application Architecture</h3> An alternative approach that people take is to instead organize their codebase around the actual application architecture. When you have put that thought in, people coming into the code base can quickly and easily understand the application architecture. They can also quickly navigate to appropriate areas of the code they want to focus on. This strategy also helps with creating logical commits, which I am a big fan of, and reducing conflicts in source control systems. People often perceive the fact that you have to think about where to put folders and files in your codebase as a negative thing. I believe this effort is one of the more valuable activities actually. It is effectively a forcing function driving you to make sure you have a sound application architecture and understand where the piece you are working on should fit within it. Using techniques like modularization and concepts like the The Onion architecture can help you figure out and evolve your application archicture and your codebase organization. But with Frameworks</h2> I know from experience fighting against a frameworks inherent design is generally not a great strategy. So how do we organize our codebase around the Application Architecture when we working with a framework like Rails or NestJS? Simple, we follow the convention of the framework up until we reach that natural boundary where it stops providing direction, e.g. the lib</code> folder. Everything inside that lib</code> folder we would organize around application architecture and not by category of architecture component. This at least gives developers the knowledge around that area of the code. Ideally it would be great if we could get those benefits at the framework level of things as well. However, it just isn't worth fighting against the framework to make it happen. Squads & Guilds 2021-05-01T00:00:00+00:00 At Uptech Studio we organize ourselves in a similar fashion to Spotify with Squads & Guilds. However, how we define Squads & Guilds is a bit different. TLDR</h2> Below is an explainer describing what Squads & Guilds are to us and some examples of how people fit into them. </a> Squads</h2> A squad is simply a group of people organized around delivering a particular project/product. They are on the hook as they are responsible for delivering the project/product to the client. Squads are formed and disbanded as needed around the various projects/products. Squad Representatives</h3> Squad representatives are a subset of the Squad that is responsible for representing the various competencies to the client. Generally there is one person from each of the necessary competencies, e.g. Design, Engineering, Product. Squad Representatives are generally comitted for the lifetime of the project as they are the face of the squad to the client. If projects are long term we may shift representives around but generally try to minimize that. Guilds</h2> A guild is a group of people focused on a particular competency, e.g. Dev-Ops, Mobile, Web, Backend, Product, Desktop, Data Engineering, Design. The following are things that guild members often help with. helping people do a certain task to hit deadlines. The assisting guild member bills their time against the project though.</li> pairing with you if you are stuck, or need to be leveled up on a tool/concept. The helping guild member should NOT bill their time to the project.</li> peer review the guild is responsible for helping you when you need a peer review and you don't have anyone with necessary competency in your Squad to do it. If you are doing the peer review, bill your time against the project.</li> manage time being a Guild member you have obligations to get your own work done. You shouldn't be doing peer reviews & pairing for other projects non-stop. So manage your time and make sure you can meet your obligations.</li> sharing you share your knowledge, tools, reusable code, best practices, etc. within the guild.</li> standards you collaborate within the guild to define standards & best practices.</li> multiple guilds you can be a member of more than one guild as you may have more than one competency.</li> </ul> Our primary communication mechanism for Guilds is Slack Channels, e.g. #mobile-guild, #backend-guild, etc. Why Squads & Guilds?</h2> Basically we needed a way to facilitate making sure we were focusing and delivering for our clients (the focus of the Squad) while also facilitating cross-pollination and elimination of siloing across the organization (the focus of the Guild). We have found this structure and the intention behind it to be extremely effective on both these fronts. Below you can see a visualization of what Squad might look like in relation to Guilds. </a> Uptech Studio Agile Development 2021-05-01T00:00:00+00:00 Kanban</h1> We are big believers in Agile methodologies. However, we don't prescribe to strict Scrum. Instead we use Kanban as we feel it better aligns with the fluidity and agileness necessary with the earlier stage companies we work with. In Kanban the columns on the board represent the prioritized queue of items that need to be done by each role in the software development process. Items move across the board until complete. We often use work in progress limits for each column to guide us around capacity issues and enforce prioritization. The Kanban boards provide all stakeholders with a clear view into the development pipeline. Boards can be customized via grouping and filtering. Columns</h2> Columns in Kanban are intended to be customized to the process at hand. By default most Kanban tools provide the following columns at a minimum. TODO - all things that the team can work on. By default, this list will continue to grow each week.</li> In Progress - all the things the team is working on</li> Done - the things the team has completed.</li> </ul> We generally evolve these on our projects to some combination of the following colmuns. Selected for Dev - Every new issue starts in Needs Review. Each week, we review this list and move issues to keep it empty.</li> Specs Needed - Stories that need further specification, UX, and/or UI assets.</li> Ready for Dev - Issues that are ready for the developers.</li> In Progress - Issues being worked on by a developer.</li> QA/Acceptance - Issues delivered by development and is now with QA and/or Product for testing.</li> Accepted - Issues that are accepted and ready for release.</li> Released - Issues that have been deployed to users.</li> </ul> Note: The above columns are guidelines we use to stay in alignment with each other. But if we are on a project where we feel like a couple of these columns aren't need and are just ritual we will get rid of those columns. Standups</h1> The Problem</h3> Uptech Studio</a> has remote employees all over the US. Therefore our team is often pretty fragmented in terms of physical location. For example, I might be on-site at a client's office while another member is at a different client's office, and yet other members are at our office. This physical fragmentation makes it difficult to stay on top of what is going on across the organization on a day-to-day basis. Normal Solution</h3> Normally, in a startup we would have a standup every morning to kick-off the day. If you aren't familiar, a standup is a time-boxed meeting that is used in many agile methodologies to facilitate communication around individuals' daily commitments, as well as raise awareness of any challenges (a.k.a. blockers). The reason it is called a standup is because the thinking is that if everyone has to stand during the meeting it would help keep the meeting short. For further details on standups and their intent check out the Wikipedia page</a>. Our Solution</h3> Given that our team is rarely in the same physical location at the same time, we have taken the principles and intent from the standup and adapted it to our situation by using a Slack</a> #standups</code> channel to share our commitments and blockers every morning at 10 am. To aid with making sure we have the standup right at 10am everyday, we set up a Slack reminder</a> in the #standups</code> channel. Then in the #standups</code> channel we communicate the classic standup information: What did I do yesterday?, What am I doing today?, Do I have any blockers? Specifically, we use the following template and generally prepare the mornings standup in a text buffer before pasting it into the #standups</code> channel. The template/example below is based on the Slack message format</a>. _Tue July 11, 2017 - Standup_ *Yesterday* _client_1_ • One thing I did yesterday • Another thing I did yesterday _client_2_ • Yet another thing I did yesterday *Today* _client_1_ • A thing I am going to do today • Something else I am going to do today _client_2_ • Another thing I am going to do today *Blockers* • A blocker that is currently preventing me from accomplishing something or will in the near future</code></pre> When the above is pasted into Slack</a> it looks as follows: Without this virtual standup every morning we wouldn't be able to successfully run our business. It constantly triggers discussions, questions, suggestions, etc. that help keep the business on track and moving in the right directions. Client Standups</h2> Not only do we post our complete standup in the #standups</code> channel to share with Uptech Studio</a>. We also post client scoped versions of our standups in shared channels between Uptech Studio</a> and the respective client, .e.g. #client1-uptech</code>. This helps in a number of ways. it helps keeps the client updated and clued into what things are progressing</li> it gives an opportunity to highlight and remind the client about blockers that they might be able to facilitate</li> it helps stay in alignment with client team and any coordination efforts between Uptech Studio</a> and them</li> </ul> The following is what the above UpTech Studio</a> standup would look like scoped down to client1</code> and would be what we would shared in the #client1-uptech</code> channel. _Tue July 11, 2017 - Standup_ *Yesterday* • One thing I did yesterday • Another thing I did yesterday *Today* • A thing I am going to do today • Something else I am going to do today *Blockers* • None</code></pre>Tools</h2> The following are a few tools we have created/found that help with this process. Though at the end of the day, like any tool/process it comes down to having an open minded collaborative team that understands the importance of process and communication to really get the value out of a practice like this. Tools are never a solution for culture but they can reduce friction. Gumleaf</h3> Gumleaf is a side project of Drew De Ponte</a> & Anthony Castelli</a> where the beta release is being shared with Uptech Studio</a> to get feedback. It is a personal task management solution built around the concept of daily standups and also provides features to trivially scope standups down to specific clients as well as export standups in the Slack format. This has become a daily driver for most of Uptech Studio</a>. Vim/Neovim</h3> Given that this is such a core part of our process we created the Vim Slack Format</a> plugin to provide syntax highlighting in Vim</a> or Neovim</a> for the Slack message format</a>. When using this plugin it looks as follows: This allows you to have a standups.slack</code> file to facilitate keeping historical track of your standups in an easily searchable format where you keep the most recent standup at the top of the file. Some people even like to keep this in a Git</a> repository so that their history is pushed up to a remote on a regular basis. Planning & Prioritization</h1> Given that we don't do Scrum. We don't really do formal "sprint planning". In fact we don't actually believe in the constraint of a "sprint". Instead we focus on "roadmap planning" and "iteration planning". The destinction is that we do plan for an iteration (roughly a week or two out). However, that planning simply funnels into the Kanban board as to what we are targeting working on. If that changes as more information comes in it isn't a problem as we are just working down through the prioritized columns. This differs from the concept of formal sprints where a sprint's tickets are usually locked in until the following sprint. We have a weekly Planning & Prioritization meetings. Planning</h3> During the Planning portion of the meeting we do the following. quickly review the roadmap against all new information and make sure it still holds, adjusting as needed</li> review all the items in Selected for Dev column and determine where on the board the tickets should go. The Selected for Dev column needs to be empty at the end of the meeting. Items can go back to the backlog if needed.</li> make sure WIP limits are adhered to and we remove items from a column to make room for other higher priority tickets when the limit has been reached.</li> </ul> Prioritization</h2> During the Prioritization portion of the meeting we do the following. review the priority of the epics making sure they still hold, adjusting as needed</li> prioritize the tickets within each of the epics covering at least a few iterations worth of work out (making sure we prioritized work in the backlog if we complete all the work planned for this iteration).</li> </ul> Attendees</h2> This meaning is owned & lead by Product but Engineers, QA, Design, etc. generally attend. Demos</h1> We generally do weekly or bi-weekly demos with the client. We use these as clear opportunities to show progress and get feedback on the work we have done for the client. Some clients like to be in the nitty gritty and get regular builds of the apps and provide feedback along the way. Others even when we provided regular builds won't really look at them until we have a demo. We do demos irrespective of the clients behavior as it is good practice to take a step back and look at the work that has been done and how it relates to the higher level product goals. When we do our Planning & Prioritization meeting each week we discuss our plans for what we are going to demo the next week. We are extremely careful not to let things be driven completely by the demo and become DDD (Demo Driven Development). But it is an important attribute to take into consider while planning and prioritizing work. We generally try and avoid demos of APIs and network requests as much as possible and focus on demos of application UI as much as possible. There are of course exceptions to this. For example if the project was to build a public facing API for a service. Then it would make sense for the demo to be Paw/Postman interacting with the public API. Retrospective</h1> We do hold Retrospective meetings bi-weekly for the squad to review how things are going. We also collect the following information from the team. what went well?</li> what didn't go well?</li> define action items with owners as a team to improve the process</li> </ul> We also review cycle time and throughput trends to make sure there are no underlying issues that need to be addressed. This meeting is owned and lead by Product. Feedback Loop</h1> In our minds Agile Development is really all about feedback loops. It is about listening, being aware, and gather feedback continuously throughout the process so that we adjust according to that feedback as we progress along. Once we start the Agile Development process doesn't mean we forgo all of the other processes we did initially (e.g. Discovery & Definition, Project Planning, Design Review, Roadmaping, Milestone Planning). In fact we continue to do them iteratively. Basically maintaining the goals of them along the way as we learn things through the Agile Development process. The way we think about it is really that we are maintaining and evolving the artifacts that came out of the initially upfront engineering process while we iterate through the Agile Development process. Uptech Studio Branching Strategies 2021-05-01T00:00:00+00:00 Branching strategies are generally applicable in two contexts. strategies used for development</li> strategies used for releases</li> </ul> Strategies for Development</h2> Our preferred branching strategy is a style of trunk-based development. This means that in terms of collaboration we have that happen over the mainline branch (a.k.a. trunk) by following a workflow we developed to facilitate the creation of small pull requests and the highest quality peer reviews. You can read more about this in our post, Journey to Small Pull Requests</a>. Behind the scenes our workflow does use branches (not feature branches) when requesting review of patches. In turn enabling the peer reviewer to review in a manner that they would be used to. This also enables us to work with teams that don't follow our workflow and allows us to still use our workflow and get the values and benefits while still playing nicely with a team that is doing feature branches. We don't believe in Feature Branches as outlined in Journey to Small Pull Requests</a> as they introduce a number of difficult problems in relation to writing quality code. We also keep a linear Git</a> history through our workflow so that it is easy to go back and revert commits, etc. if necessary. Strategies for Release</h2> Many people use a branching strategy for releases. For example they might have a production, qa, and dev branch mapping to respective environments. We do NOT use environment based branching or any other branching strategy for releases generally. We instead use Git</a> tags to manage releases. This enables us to have and maintain our linear history while actually using immutable tags to represent releases and in turn trigger automated builds and deployments. In some cases we might also choose to do manually triggered releases. Sometimes we are stuck in a client's environment in which they have built up a bunch of automation around environment based branching. In those case we generally recommend that client make their automation work in such a way that they don't have to merge branches back and forth and instead the branches can be force pushed to, triggering automation and deploys. This helps mitigate a bunch of confusion within the Git</a> history. Uptech Studio Error Handling 2021-05-01T00:00:00+00:00 Our general position on error handling is that we should be using Result</code> types with base exception classes so that we can easily transition between the Result</code> type world and the Exception</code> worlds. The following is a presentation Drew De Ponte</a> gave that covers this topic and how we should be thinking about error handling. </iframe> Abstractions & Errors - Slides</a> TypeScript Example</h2> The following is an example of how we can create a Base exception in TypeScript and how we can create Custom exception types and use them with Result</code> types and how we can programmatically handle the cases. import { Ok, Err, Result } from 'ts-results'; // Here we define a CustomError that all custom errors should extend from so // that the custom errors don't have to worry about setting their name // appropriately. class CustomError extends Error { constructor(message: string) { super(message) this.name = this.constructor.name; // This is something that we have seen on the web for issues where Babel won't // set the stack trace properly when the class doesn't extend properly. But // from our testing this hasn't seemed to be needed and it is also our // understanding that super() now does this. // // if (typeof Error.captureStackTrace === 'function') { // Error.captureStackTrace(this, this.constructor); // } else { // this.stack = (new Error(message)).stack; // } } } // This is an error type that is useful to have around because if you have a // function that returns Result<someType, yourErrorType> you don't want that // function to throw an exception because you are using Result. So inside // the function you would do something like the following: // // try { // ... // } catch (e: any) { // return new Err(new UnhandledError('some message', e)) // } // // This allows you to identify it is an unhandled error but still get to the // internal error object if needed later on. class UnhandledError extends CustomError { origError: Error constructor(message: string, origError: Error) { super(message) this.origError = origError; } } // The following are examples of defined custom error types as a hierarchy // for categorization. In this example we have a DashboardEventsError as a // base category for the errors and then we have MyErrorClassA and // MyErrorClassB as specific errors within that category. class DashboardEventsError extends CustomError {} class MyErrorClassA extends DashboardEventsError {} class MyErrorClassB extends DashboardEventsError {} // The following is code I used to vet the functionality of the custom errors // and make sure that they meet all the expectations of the interface for the // Error type const e = new MyErrorClassB('some message'); // - can we get the name and is it the correct custom name console.log("name: ", e.name); // - can we get the message console.log("message: ", e.message); // - can get stack trace console.log("message: ", e.stack); // - can we conditionally check the custom type to programmatically handle specific errors if (e instanceof MyErrorClassA) { console.log("it is an instance of MyErrorClassA"); } else { console.log("it is NOT an instance of MyErrorClassA"); } if (e instanceof MyErrorClassB) { console.log("it is an instance of MyErrorClassB"); } else { console.log("it is NOT an instance of MyErrorClassB"); } // - can conditionally check if it is an instance of custom base class type if (e instanceof DashboardEventsError) { console.log("it is an DashboardEventsError"); } else { console.log("it is NOT an DashboardEventsError"); } // - can conditionally check if it is an instance of Error type if (e instanceof Error) { console.log("it is an Error"); } else { console.log("it is NOT an Error"); } // - do we get the custom type name and stack trace when it is thrown throw e; // Sadly because of limitations of TypeScript I haven't found a good way yet // to meet the following criteria. // - communicating to the user via the type system what the possible error types are // - requiring handling of all possible returned errors or explicitly declare not handling // // The following are just explorations I started but didn't really go anywhere to meet those requirements // - communicating to the user via the type system what the possible error types are // // typescript doesn't seem to complain even through I declared the possible // return types that I am returning a different error type. So not really // useful as a tool for the author of the function. Maybe it could be seen as // documentation for the consumer function foo(): Result<any, MyErrorClassA | UnhandledError> { try { console.log("in try"); return new Err(new MyErrorClassB('error a')); } catch (e: any) { console.log("in the catch"); return new Err(new UnhandledError('message', e)); } } // - requiring handling of all possible returned errors or explicitly declare not handling // // I don't believe there is anything in TypeScript to support exhaustive // handling of error types. I have contemplated having a pairing function with // each function that enforces this. However, there is nothing to help the // author of that function make sure they aren't forgetting any error types. // Also it doesn't help the use case when you are mapping errors in the middle // of a chain. // // So I think for the time being unless we were to look into a TypeScript // compiler extension we are stuck in this less than ideal world. let result = foo(); if (result.ok) { console.log("result ok"); } else { console.log("result err"); if (e instanceof MyErrorClassA) { console.log("it is an instance of MyErrorClassA"); } else { console.log("it is NOT an instance of MyErrorClassA"); } if (e instanceof MyErrorClassB) { console.log("it is an instance of MyErrorClassB"); } else { console.log("it is NOT an instance of MyErrorClassB"); } }</code></pre>Envolving Classified Error Classes</h2> Creating unique error classes for each specific function will result in a lot of different error classes. However, we need to make sure that we formalize explicitly all the possible errors that can result from a function. One obvious way to collapse the number of errors is to make Errors more generic. For example you could have a FileNotFound</code> error for one function and you can also have another function use the same FileNotFound</code> error. How do you know how to make an error generic or not or if you make it too generic. I have found that defining the error classes with their associated data helps a lot. It gives you a guide to understand if the associated data makes sense in the other generic cases as well. So the recommendation is to start wtih a specific error definition with associated data and then extrapolate from there into a generic type that can be reused. FAQs</h2> Is it common for nearly every function in an application to return a Result to handle potential errors? At which point, if any, would that become unnecessary? It seems like this has become the ideal solution for dealing with breaking errors in the Trans-Serv, but I don’t want to find myself in a position of over-using or defaulting to Results if that is not generally the ideal way to design/build applications. In reference to, "Is it common for nearly every function in an application to return a Result to handle potential error?" The answer is generally yes it is common to use it everywhere and you generally should. The point at which it becomes un-necessary is the point at which an error is handled programmatically. Usually this is at the bounds of the application (e.g. user interface where error is presented to the user, or where an error is internally programmatically handled). You should be using Result whenever it is possible for the operation you are performing to fail. The only caveat to this is if you are in an async chain (aka a promise chain) and you want a function to throw an exception explicitly for being used in that context of an async chain. However, generally in these cases the async chain is basically mapped back to a result at the end anyways. At what point does piping, mapping, or promise chaining become “too large”? Is the limit determined on the amount of the logic inside each consecutive call? For example, would it be easier to follow a larger number of chains if it’s purely passing in the next function, as opposed to having chains where logic must be implemented beyond calling a single function (even if it is only a few lines)? Is the latter case a “smell” that the logic should be extracted into a new function to cater to the former? In reference to "At what point does piping, mapping, or promise chaining become “too large”?" This is a subjective thing and hard to get away from it being subjective because the driver of it is readability and being able to easily understand what is going on in the chain. If I have to read through a chain of 20 different steps to understand what the entire chain is trying to accomplish it is too large. I should be able to quickly look at it and understand and somewhere around up to 5 steps or so is a good rule of thumb. It is worth noting though that if you have a really large chain, you should be collapsing multiple steps into a single step with a name that has meaning using pipe(). Having chains where you hand it a lambda is a smell. The ideal is to be in a situation where your chain is completely composed of named functions. This drastically aids with the understanding of the chain and its intent without having to worry about the details of understanding the implementations of those lambdas. If you feel like you have to do logic beyond a single function it is a "smell" that, that logic should be extracted out into a new function. Ok so new question relating to error handling. For the stream creation helper function, Jon suggested that we return a FileNotFound error to be specific if a file is not found. The way our Error handling works currently, everything bubbles up to the TransactionServiceError class which then emits the error with an http error code - if I understand correctly. If an internal function is using the helper function to transform a file path into a stream, we would want to throw a FileNotFound error - but it would not be an http error of 404, it would be an internal error. When handled by the calling function it might be transformed into a 404, depending on the context (e.g. if it is interacting with the outside and needs to communicate that the file path is missing), or it might remain an internal matter to be passed along as a generic 500 error for the greater service. So how do we handle this sort of instance? I created a FileNotFound error so it would contain the message to be handled internally, and it could theoretically be caught and transformed into a TransactionServiceError code 404 by something that handles it... but I am not sure if that is how we want to manage this? Im not certain we want everything to be a TransactionServiceError right, since not everything has an http code. Functions can and should have their own errors to inform the caller of the status via Result. TransactionServiceError was designed in relation to Faktory Jobs consuming failures when calling routes, which is separate from this. The potential transformation is outside the scope of this context. This FileNotFoundError</code> could be transformed by the caller (if needed). I would ideally like to catch any errors that might occur when creating the stream - in a try catch or something similar. Since it's a stream it is a bit more complex, since it has an event to listen to for errors .. which I assume the handler would want to handle in this case. But in general, if I were to need to catch unexpected errors how would I manage an exception that doesn't conform to the Result signature here? Or should I use a more generic Result signature with Error as the Error type and just pass up exceptions as wrapped errors in the result, for e.g. (and the missing file error would still be of type FileNotFoundError) The function should try to do its best to prevent rouge exceptions. What we have here in this example is good and if there is a rogue exception then the caller should handle it or not. I dont like the idea of a generic Error type in the signature since its unclear of the intent of the function. Rogue exceptions are well that, rouge. We should do our best and the example code above seems to do the best it can. I’d say make a judgement call and we can go from there. The one addition I would add to that is that you can handle "rouge" / unhandled errors not by generalizing the return type to use Error but instead by formalizing the concept of Unhandled error as part of the valid error type in the return type. Uptech Studio Git Commit Standards 2021-05-01T00:00:00+00:00 The Git commit is probably the most fundamental unit we have to manage the growth and evolution of our source code. Git commits can be extremely useful sometimes and other times they can be useless. The following are the guidelines we have honed in on over the years to help make sure that our commits end up being the valuable kind. Commit Characteristics</h2> To aid with making sure our commits are the valuable kind and ensure they play nicely in the Git Patch Stack</a> and Continuous Integration methodologies we use the following characteristics as a checklist. logical</li> buildable</li> testable</li> not-necessarily complete</li> releasable</li> </ul> Logical</h3> Each commit should be a logical unit. This means it should include only changes related to that logical unit. This includes related code changes, automated test changes, tooling changes, etc. This is crucial to support the ability to easily revert changes when necessary. It is also crucial for aiding in doing any sort of git commit tree spelunking. Buildable</h3> Each commit should be buildable. This means that each commit must NOT break the build tooling or build process of the application. This is important to make sure that all the Git tooling (e.g. git bisect) continues to be able to be used. It also more clearly supports Git tree spelunking and hot fixing as no matter what commit you pick it should be in a buildable state. Testable</h3> Each commit should be testable. This means that each commit must NOT break the automated testing tools or the tests. Similar to buildable this is important to make sure that all the Git tooling (e.g. git bisect) continue to work. It also helps with hot fixing, etc. Not-Ncessarily Complete</h3> Each commit does NOT have to be complete. This means that you could for example have a commit that introduces a new method to an existing class. That is a complete logical change that is buildable and testable. But it isn't complete in terms of its interactions with other application architecture concepts. This is really just a reminder that we should NOT be including tagentially related changes into our logical commits. Releasable</h3> Each commit should be releasable. This means that once that commit is integrated into mainline. We should be able to cut a release of the application without a problem. So it shouldn't break the release process and it shouldn't leave the user experience of the application in a broken state. Most of the time you will find that commits can just naturally be releasable. But sometimes it means that you have to create initially unused alternate code paths or use feature flags/toggles to manage code paths so that code can be integrated into mainline while still keeping the commit releasable. Git Patch Stack</h2> To help us follow these best practices we have developed a workflow on top of Git called Git Patch Stack</a> and built a small command line Git extension to help streamline it. You can get a deeper understanding of this workflow and how to use it via our blog posts, Journey to Small Pull Requests</a> and How we should be using Git</a>. A Communication Tool</h2> Just as import if not more important than the source code change is the commit message itself. This is the only mechanism a commit author has to communicate very important things like, what, intention, reason (a.k.a. why), approach (a.k.a. how), all crafted with context. Having this crucial information bound to the source code change is amazingly powerful and valuable when developing and maintaining an application over time. We have found it is also a useful communication tool for some automation tasks. Specifically for including Changelog entry information so that we don't create unnecessary conflicts in the commit itself. Git Changelog</a> can then use this information to generate our Changelog files for us. Official Git Commit Message Format</h3> Git as a tool has guidelines for commit messages to work best with git and its various tools. This format is as follows: Short summary of change (50 charecters or less in length) One ore more paragraphs of text hard wrapped at 72 characters in length.</code></pre>Our Git Commit Message Format</h3> We have built on the Official Git Commit Message Format to make sure we include this extremely important information. The following outlines our format. Short summary of change (the what) - 50 characters or less in length One or more paragraphs explaining the INTENTION & REASON of the change with valuable context. (Hard wrapped at 72 characters) One or more paragraphs explaining the APPROACH taken to attempt to achieve the INTENTION or a portion of the INTENTION. (Hard wrapped at 72 characters) Associated Ticket Identifers Git Changelog header and entries if applicable</code></pre>Examples</h3> Definitely NOT what we are looking for</h4> Intergrate Sentry for crash handling</code></pre> or even worse Update project.pbxproj</code></pre> Picture someone throwing this commit over a wall to you with no other information and they asked you to review it. What would you actually be able to say about it other than what is already included in the diff. You could maybe call out a small syntactical, style, or maybe some logic things strictly based on the diff itself. But you wouldn't be able to provide any value in the peer review on if the changes meet the intent because you don't know what the intent is. You can't analyze the approach and provide valuable analysis of it, nor can you understand if the change is the right change to make to reach the over arching reason for the change. We don't even know from the short summary provided what in the project.pbxproj file changed. As a result you'd have to read the entire diff of changes and infer the intent, approach, and reason. Making logical, purposeful, buildable, testable, not-necessarily complete, but releasable commits with verbose commit messages may seem like overkill, but it becomes immensely useful for all the reasons stated in this guide, and more important when you no longer have access to the original developer who made the change! There are even benefits seen when the developer can't recall the intent, approach, and reason of changes that were made only a couple months ago. What we are looking for</h4> Intergrate Sentry for crash handling We did this because we previously had to get rid of MS AppCenter because it wasn't compatible with mac Catalyst. The belief is that Sentry is actually compatible and may be a better solution for us with the need to support mac Catalyst. Also, living with out crash reporting is just not smart. This involved making a build configuration change to the Debug Information Format changing it from `dwarf` to `dwarf-with-dysm` so that we could also add a build phase script that would upload the build's dsyms to Sentry so that it could symbolic the backtraces. Beyond that we registered the sentry API tokens and enabled the crash handler at app launch. Note: We did this with two different Sentry projects, one for the mac Catalyst build and the other for the iOS (iPad/iPhone) build. This is necessary because they are separate builds that get deployed separately.</code></pre>Commit Message Template</h3> If you are having difficulties remembering the git commit message format and what you need to provide in your message you are in luck. Git provides a commit message template feature. You can set it up with the follownig steps. Create your template file in a safe place with the following content. Common practice is to put it in ~/.gitmessage</code>. # Short summary of change (the what) ( <= 50 chars ) # INTENTION of the change & REASON for change # (Hard wrapped at 72 chars) # APPROACH taken & WHY in relation to attempting to achieve INTENTION # (Hard wrapped at 72 characters) # Deployment or Rollout Considerations # Automated Tests considerations - if **NOT** why? # Associated Ticket Identifers # Git Changelog header and entries if applicable # [changelog] # added: some addition you made that you want in your changelog # changed: some change you made that you want in your changelog # deprecated: some deprecation notice you want in your changelog # removed: some removal you want in your changelog # fixed: some fix you want in your changelog # security: some security fix you want in your changelog</code></pre></li> Configure Git to use your new commit message template$ git config --global commit.template ~/.gitmessage</code></pre></li> Make commits without (-m</code>) and it will bring up the commit message template in your editor.</li> </ol> Uptech Studio Peer Reviews 2021-05-01T00:00:00+00:00 The intent of this document is to provide a reference document for the Peer Review Process that we follow. Goals</h2> The following are the main intentions of this process. improve code maintainability</li> reduce defects in production</li> developer growth</li> </ul> Participants</h2> All developers, no matter skill level, should participate in the peer review process. If only a subset of the team does the peer reviews, this bogs down the more senior people with a disproportionate amount of work. The senior people usually are already spending time helping others, so we don't want to put even more work on them. Additionally junior people can benefit from looking at senior developer's work. They can see what the code should be like and learn tips/tricks. Mindset*</h2> Please be open to suggestions from your peer reviewers and don't take things personally. On the flip side, please be aware that someone has put in a lot of effort into their code and sometimes there can be multiple viewpoints. In order to have a better collaborative experience for all people involved and still achieve the stated goals, please consider the following suggestions: Ask questions: How does this method work? If this requirement changes, what else would have to change? How could we make this more maintainable?</li> Compliment / reinforce good practices: One of the most important parts of the code review is to reward developers for growth and effort. Few things feel better than getting praise from a peer. Try to offer as many positive comments as possible.</li> Discuss directly for more detailed points: On occasion, a recommended architectural change might be large enough that it’s easier to discuss it in person rather than in the comments. Similarly, if you are discussing a point and it goes back and forth, just pick it up in person and finish out the discussion.</li> Explain reasoning: It’s best both to ask if there’s a better alternative and justify why you think it’s worth fixing. Sometimes it can feel like the changes suggested can seem nit-picky without context or explanation.</li> Make it about the code: It’s easy to take notes from code reviews personally, especially if we take pride in our work. It’s best to make discussions about the code than about the developer. It lowers resistance and it’s not about the developer anyway, it’s about improving the quality of the code.</li> Suggest importance of fixes: Not all suggestions made need to be acted upon. Clarifying if an item is important to fix before it can be considered done is useful both for the reviewer and the reviewee. It makes the results of a review clear and actionable.</li> </ul> Process</h2> Developer Responsibilities</h3> Developer makes a patch</li> Developer runs all automated test suites and makes sure it's green, or relies on CI server to do so and awaits green response.</li> Developer manually tests the feature to see if acceptance, functional, ux, and ui criteria are met.</li> Deveolper Requests Review of their patch Note: at this point the pull request should meet the Pull Request Standards</a></li> If a patch is not complete, but you want feedback on it earlier in the process, you can Request Review of it, but indicate clearly in the title and description that the patch is not complete and not ready to be merged and what you're looking for feedback on.</li> </ul> </li> </ul> Reviewer Responsibilities</h3> Peer Reviewer assigns themselves to the pull request.</li> Peer Reviewer checks out the code</li> Peer Reviewer runs automated tests, if they fail, deny the pull request.</li> Peer Reviewer does manual testing Does it meet the acceptance criteria identified in the ticket?</li> Are there any performance issues?</li> </ul> </li> Peer Reviewer reads through the diff and comments on it. (see What to look for when reviewing below)</li> Peer Reviewer approves/rejects the PR depending on their findings.</li> </ul> More Developer Responsibilities</h3> Once approved, the Developer should then publish the patch to close the pull request and integrate the code. The peer reviewer should NOT merge the code as the developer knows best if it should be integrated into mainline or not.</li> Developer should cleanup their remote branch associated with their PR.</li> </ul> What to look for when reviewing*</h2> High Level</h3> Don't Repeat Yourself (DRY): If you see a block of code that is duplicated, wrap it in a function and re-use it.</li> Code left in a better state than found: If you are changing an area of the code that’s messy, it’s tempting to add in a few lines and leave. Go one step further and leave the code nicer than it was found.</li> </ul> Style</h3> Commented code: Good idea to remove any commented out lines.</li> Function length: Rule of thumb is that a function should be somewhere around 1-10 lines in length. If you see a method above 10 lines, it’s likely needs to be broken down into smaller pieces</li> </ul> Testing</h3> Test coverage: All new features and bug fixes should have tests. Are the tests thoughtful? Do they cover the failure conditions? Are they easy to read? How fragile are they? How big are the tests? Are they slow?</li> Number of Mocks: Mocking is great. Mocking everything is not great. Use a rule of thumb where if there’s more than 3 mocks in a test, it should be revisited. Either the test is testing too broadly or the function is too large. Maybe it doesn’t need to be tested at a unit test level and would suffice as an integration test. Either way, it’s something to discuss.</li> Meets requirements: Take a look at the requirements of the story, task, or bug which the work was filed against. If it doesn’t meet one of the criteria, it’s better to bounce it back before it goes to QA.</li> </ul> * some items taken from: Code Review BestPractices</a> Uptech Studio Process 2021-05-01T00:00:00+00:00 Discovery & Definition</h1> One of the first phases we go through in a project is what we refer to in the engineering space as Discovery & Definition. The goal of this phase is to get a decent understanding of what the project should be and what the initial scope of the project should be. This is generally done through a series of meetings between the stake holders and Uptech Studio Product, Engineering, and Design representatives. This is where we combine what we know the client wants with our knowledge and expertise to eventually result in what we are actually going to build for them. This is also often where we pose questions or push back on the client to identify if ideas have been vetted and tested to make sure that their money will be well invested. Once we have these meetings and have come to a conclusion as to what the product should be and what our initial scope will be we move onto the next phase, Estimations. Estimations</h1> Prior to this Product is responsible for outlining User Stories based on the learnings from the Discovery & Definition phase. Here at Uptech Studio we aren't big believers in estimates. In fact we generally don't believe or trust in estimates. But our clients need to be able to budget and plan, not down to a date or week even. But they need to have an understand of relatively large time range and budget range to find out if they can even afford to pay us and if the timeline will work for their business need. So we use this as an opportunity to have the Tech Lead start to think through all the stories and what is going to be involved with them and estimate the story at the scale of indivdual developer worker days. This is generally done in Airtable. We then sum up all the working days and add a multiplier based on confidence and unknows using around 1.5 and 2 to account for the fact that estimates are generally over optimistic. These numbers can then be used to compute roughly time range and cost range based on these estimates. Project Planning</h1> The Project Planning phase is where we take the knowledge we gained around the project scope and start planning out the high level technical details around the work. The goal of this phase is to do an initial relatively high level of planning to start to breaking down the technical problem space and identify unknowns, concerns, and blockers. This is generally achieved with the following artifacts of this process which are the responsibility of the Tech Lead to produce. Systems Architecture</li> Project Detailed Spec</li> Defining Unknowns</li> </ul> Systems Architecture</h2> The first artifact for this phase is a Systems Level Architecture diagram that the Tech Lead is responsible for putting together. The goal of this is to simply start the conversation and discussion around all the entities in play at a system level and how they are going to interact with eachother. Over time this continues to be updated and used as a conversation piece to discuss and review systems level interactions. Project Detailed Spec</h2> The Project Detailed Spec is a document that the Tech Lead is again responsible for. This document defines the project scope and goals for review as well as the Systems Architecture to provide context. Beyond that it is a vehicle to explore the next tier down of technical problem solving. This could be how the various components would be interacting with each other. Is it done via GraphQL, REST, etc? What are some rough ideas of what the key functional data is needed to be passed between these system components. These types of questions and thinking. It isn't to formalize 100% exactly what the payloads will look like. It is just to get an idea and an understanding to gain more confidence. The process of the Tech Lead walking through, planning this out and documenting it helps them start to think through all of the components and how they will interact to accomplish the defined goals. This also helps identify up front major unknowns, concerns, and blockers very early on in the project so that those things can be focused on from the get go. Project Design Review</h2> Once the Project Detailed Spec is produced the Tech Lead then schedules a Project Design Review internal to Uptech Studio. This meeting enables a round of review and feedback on the architecture and design that is documented in the Project Detailed Spec. This generally should have a representative from Product, Design, and Engineering. The people at this point aren't from your Squad. As a Tech Lead you can choose to do Squad internal rounds of feedback with the engineers prior to this meeting. Project Kick-off</h2> Once the Tech Lead has held the Project Design Review, gotten feedback, and integrated that feedback into their Systems Level Architecture diagram and Project Detailed Spec they then schedule a Project Kick-off meeting with their Squad. This involves all members of the Squad and provides in opportunity to go over everything covered in the Detailed Spec from the project Goals down to how the various components will be interacting and with what data. The purpose of this meeting is for all the members to get comfortable with all of this and voice any concerns, suggestions, etc. Upfront Planning?</h2> I thought Uptech Studio did Agile. We do! Nothing in Agile says that you shouldn't do up front planning. Agile is all about iterating and getting feedback throughout the process and adjusting to that feedback. But nothing in it says you shouldn't think about how you would build or implement something. Our experience over the years has informed our opinion that doing a small amount of up front planning like this is often a game changer in terms of the success of a project. Design Review</h1> This document describes how and when a software design review should take place. What is it?</h2> A design review is an opportunity to get feedback on your application design prior to spending a lot of time working on the implementation. Even if you love to do emergent design using TDD, when projects are large and complex, some upfront design is necessary. It's hard to identify all the components that exist and what their interactions are with existing systems without taking some time to think through it. It's also really helpful so you can break down a larger project into smaller components. As a general rule of thumb if a project is going to take more than a week to develop, then it probably should have a design review. Smaller projects will have real quick meetings, whereas larger projects will probably take some serious brain power. When does it take place?</h2> A design review should happen after the requirements have been documented and delivered to the implementation team, but before any code has been written. Ideally the product owner will have a kickoff meeting with the team to go over this new feature and how it should work. After that, the engineers can review the requirements in detail and start coming up with an implementation plan. Within a few days of the kickoff meeting, the engineers should have a design review meeting to go over the plan. Who should be part of it?</h2> The technical lead of the project is responsible for scheduling time to plan and document the proposed design. They are also responsible for scheduling the review meeting and making sure they have proper representation at the meeting. A design review should have the following people: Technical Lead</li> Backend Representative</li> Front-end Representative</li> Infrastructure Representative</li> QA Representative (Ideally the person that will be testing this feature)</li> </ul> Including members from other teams helps to address any concerns they may have earlier in the process as opposed to later when it's more expensive to change. How does the meeting run?</h2> During the meeting, the technical lead on the project should review the feature being implemented at a high level so everyone in the room understands what the business objectives are. After that, they should walk through a document detailing the implementation plan. The document should include the following: A high level description</li> A diagram of the different systems/components</li> A sequence diagram indicating interactions between components/systems</li> Interface specifications (REST API, Message Formats, etc.)</li> Callout for new Infrastructure components</li> Rollout plan - call out any sequencing required</li> </ul> Why should we do this?</h2> Because we want to build awesome products with the highest standards in mind and avoid making desicions that may bite us in the future. Roadmapping</h1> The Roadmapping phase is where the Tech Lead and the Product representative collaborate together to define a Roadmap of Milestones for the project. During this process we take a number of things into account. technical dependencies & constraints</li> best way to show client progress without fabricating Demos or doing Demo Driven Development.</li> make sure we can internally deliver testable versions to stakeholders</li> </ul> This is done in Jira using the Roadmap feature. The high level Milestones are broken out and defined & prioritized first based on the criteria above in collaboration with Product. The Product person then takes the User Stories from the Estimations phase, cleans them up (adding any new ones, adding details to stories, etc.), and organizes them into the roadmapped milestones. Milestone Planning</h1> The Milestone Planning process is basically the same process that we do in terms of Project Planning. The main difference is that we are planning within the scope of a defined Milestone. This process is again driven by the Tech Lead. It can be thought of in the following sections. Scoped Systems Architecture Diagram</h2> When doing Milestone Planning you are scoping your thought process and in turn your created artifacts to a particular milestone. Therefore, you are creating a System Architecture Diagram that only focuses on the needs for this particular milestone. This is useful again to provide context but also useful to remove the noise of everything outside of the scope of this Milestone. Application Architecture</h2> The Application Architecture is where you start to visually layout how your team is going to build an application that will meet the needs of the goals. This again is about components, how data flows through them, and what components handle different types of processing. Scoped Protocol Specifications</h2> These protocol specifications again aren't inteded to be 100% accurate in terms of every piece of data. It is intended to be a tool to make the Tech Lead think through the core data that needs to be exchanged via this different channels and what those protocols would look like. Infrastructure Diagram (depends on need)</h2> Infrastructucture Diagrams aren't always necessary. Basically if there is something complex and abnormal necessary in the infrastructure then we should create an infrastructure diagram. The other use case might be that a client is explicitly requesting or pushing for an Infrastructure Diagram. Sequence Diagram (depends on need)</h2> A Sequence Diagram is again dependent. If you are in a situation where you need to clearly define and communicate about complex interactions between system level or software level components a Sequence Diagram is the tool for you. Flow Chart (depends on need)</h2> Flow Charts again aren't required but are super useful when you have some complex logic that needs to be planned out and thought out. Detailed Design Spec</h2> The Tech Lead then puts together a Detailed Design Spec similar to the concept and structure of the Project Design Spec from before except scoped down to the concerns of this one milestone. Unknowns</h2> Part of this process is identifying unknowns. We use these unknowns to guide us by effectively being blockers that need to be figured out to complete this milestone. We need to try and answer these unknowns as much as possible prior to the design review. However, some unknowns might be dependent on getting feedback from 3rd party vendors, etc. We should start those conversations with product and the 3rd party vendor ASAP, prior to the design review. This list of unknowns should be a working list of blockers that should get formalized as tickets and prioritized generally to be figured out ASAP. There are sometimes cases where you might want to not prioritize resolving one of these unknowns until later. This should be vetted with the project's principal engineer. Design Review</h2> Once completed the Tech Lead schedules a design review (same process as before) except this time it happens within the Squad. If the Squad feels they need external imput from people outside the Squad they are welcome to invite other people. Kick-off Meeting</h2> Once the Design Review process is complete. The Tech Lead schedules a kick-off meeting for the Squad. This has the same purpose as before. It is to get everyone comfortable with everything and provide a venue for people to raise concerns, ask questions, etc. Task Breakout</h2> One thing that different Squads tend to do differently is task breakout. Some Squads have an extra step here after the kick-off meeting where they schedule a meeting with Squad Product representative and the Engineers on the Squad and go through the tickets in the Milestone and breakout sub-tasks for each of the stories. Other Squads don't do this meeting and instead leave the task breakout to the Engineer that grabs the story. We don't have strong opinions on this. It is really whatever the Squad is comfortable with. The key is that our Squads are constantly self evaluating so if there is a problem in this area they can quickly adjust. Milestone Estimations</h1> Despite our dislike for estimates due to their inaccurate nature we still do them in some form because the reality of the situation is that we need to know relatively early in the process if we are going to be able to hit a timeline or not. Obviously we can't know this 100% because as we have stated estimates are very inaccurate. However, they can gives us a pulse to know if we think we are significantly going to miss a timeline. This is crucial as that pulse is what instigates pruning of non-crucial features and setting expectations appropriately with the client. To facilitate this we of course start with the initial estimates from the Discovery & Definition phase. However, things have likely changed since that time and the developers haven't thought about the complexities of each of those milestones. To facilitate this when a Journey is reviewed by product with the associated dev team. The dev team goes back through the milestones of that Journey and puts together estimates based on the following thinking. Origination of Thinking</h3> Historically we have focused on the concepts of Complexity and Effort while putting together estimates and it was only our Milestone Planning that thought about unknowns and risk. However, relatively recently Theo dropped a video where he broke down what he has thought of for a system for estimates. It is very much in alignment with our current approach as it takes Complexity and Effort into consideration. However, beyond that it takes into account Risk and goes through the reasons for estimating in a more analytical fashion. Therefore, we have effectively adopted his approach with some small tweaks. We also differ from his approach in that after thinking through these characteristics we then estimate how long things will take (in developer days) using these characteristics guidence. This is critical for us to understand how we are making progress in terms of our estimates so that we can make adjustments and manage client expectations appropriately. Theo - Why Are Estimations So Hard??? PLANNING TASKS AS AN ENGINEER</a> Why we estimate?</h2> We do estimates at the epic level for the following reasons. Identify who should do it</li> Identify when it should be done</li> Identify low performers</li> Bring risk to the forefront so that it can be addressed/mitigated</li> Prioritize based one ^</li> Identify how long it will take (this will be wrong, but hopefully taking the following system into account will help make it less wrong)</li> </ul> The Breakdown</h2> Our estimation system is broken down into 3 categories, Complexity, Effort, and Risk. Complexity (1-10) - How "hard" is this thing to do?</li> Effort (1-10) - How much work does it take?</li> Risk (1-10) - What are the chances that Complexity or Effort is off?</li> </ul> Getting Started</h2> When first getting a team started thinking in this way we have found that it is useful to often go down to the story level and document the Complexity, Effort, and Risk as it ends up being a forcing function to think about the complexities and risk. Then go back up to the milestone and produce the milestones Complexity, Effort, and Risk. Then you can look at the stories overall and validate that your estmiate for the milestone makes sense. Once the devs start to get the feeling for thinking about Complexity, Effort, and Risk this process goes really quickly. Breaking out Research Tasks</h2> As we go through this process we are often running into Risks and most of the time you need to do some further research to understand and mitigate the risk. This is the perfect time to break those tasks out. You can also see by doing so how breaking the research task out helps you to think about and adjust the characteristics for the story and its associated research task. Example</h2> The following are a couple examples from the video referenced above with the addition of the developer days estimates. Display voice levels in pre-call view Complexity: 4/10</li> Effort: 2/10</li> Risk: 6/10</li> Estimate: 4 dev days</li> </ul> To reduce the risk of this task we could create a research task to figure out one of the bigger pieces of risk. For example. Identify how to get voice levels from 3p SDK Complexity: 2/10</li> Effort: 2/10</li> Risk: 4/10</li> Estimate: 1 dev day</li> </ul> Uptech Studio README.md Standards 2021-05-01T00:00:00+00:00 We require README.md</code> files in our repositories so that people can understand the following. what the repository is</li> how it fits into the bigger context</li> how to get the development environment up and running</li> how to contribute to it</li> how to run the test suites</li> how to deploy it.</li> </ul> README.md Content</h2> The following is an example of a README.md</code> file and its content. Use this as a template. # git-ps - Git Patch Stack Git patch stack is an add-on command for Git that facilitates using a patch-stack [Git][] workflow while still using pull requests for peer review. It accomplishes this by managing your stack of patches and behind the scenes creating branches for patches you request review on and pushing them up to origin. It consists of the following commands: - `git-ps ls` to list the stack of patches - `git-ps show <patch-index>`to show the patch diff and details - `git-ps pull` to fetch the state of origin/master and rebase the stack of patches onto it - `git-ps rebase` to interactive rebase the stack of patches - `git-ps rr <patch-index>` to request review of the patch or update existing request to review - `git-ps pub <patch-index>` to publish a patch into upstream's mainline (aka origin/master) - `git-ps --version` to output the version of information for reference & bug reporting ## Installation If you are on a platform other than macOS you will have to build your own version from source. ### macOS To install on macOS we provide a [Homebrew](http://brew.sh) tap which provides the `git-ps` formula. You can use it by doing the following: #### Add the Tap brew tap "uptech/homebrew-oss" #### brew install brew install uptech/oss/git-ps ### Build from Source If you are on another platform you will have to build from source. Given that `git-ps` is managed via [GNU make][]. It can be built as follows: $ make build Once you have built it successfully you can install it in `/usr/local/bin` using the following: $ make install ## Development We use [GNU make][] to manage the developer build process with the following commands. - `make build` - build release version of the `git-ps` - `make install` - install the release build into `/usr/local/bin` - `make uninstall` - uninstall the release build from `/usr/local/bin` - `make clean` - clean the build directory ## Recommended Git Configuration People using the patch-stack workflow often enable the pull-rebase option in their `.gitconfig` as follows. **Note:** This is **NOT** required as `git ps rebase` is used generally. However, it is recommended so when you play outside of `git ps` you get the same awesome benefits. [pull] rebase = true This makes it so that when you do a `git pull` it will automatically trigger a rebase on top of the tracked branches remote. So, if you are using `master` and you do a `git pull` to get recent changes from `origin/master`. [Git][] will automatically initiate a rebase of your changes that were sitting in `master` ontop of `origin/master` on to the new `origin/master`. This is exactly what you want as conceptually your stack of patches are always sitting on top of the latest `origin/master`. ## What is a patch-stack workflow? The patch stack workflow is a different mental model for doing development locally. Instead of having feature branches and doing work inside them you generally do all your work directly on `master` by convention. You can think of your local `master` being a conceptual stack of patches (a.k.a. commits) on top of `origin/master`. The idea is that instead of creating feature branches you simply make commits on your local `master` and perform interactive rebases to squash, reorder, ammend, etc. your patches as you continue to develop. This has an amazing benefit. **It eliminates the dependent branch problem.** You know the one where you created one change in a branch and submitted for review, but then you went off to create another branch on top of the last one, and so on. Soon enough you have to make changes as a result of the review to the first feature branch. And, down the feature branch rebasing train you go. The Patch Stack workflow also expects each patch (a.k.a. commit) to be **small, buildable, logical units of work with valuable messages** providing the *what*, *why*, and any *contextual details around how*. This aids drastically in the code review process as each patch becomes a very focused piece of work with clear intent. It also helps conceptually "show your work", the logical progression of changes (patches) needed to accomplish some larger goal. These principles are as extremely valuable when doing historical spelunking to learn a code base or to iron out a bug. These principles bring even more value as they enable you to take advantage of all the amazing features of [Git][], e.g. `git bisect`, etc. ## Common tasks git-ps assists with ### Visualizing your Stack of Patches [git-ps][] provides the **list** command (`git ps ls`) to visualize your current stack of patches and the state around each of the patches. The output of this command for the example above would look something like the following. 2 4387d1 add login view 1 23d7a6 add login support to API client 0 af73d2 add lib to facilitate making REST API requests This outputs what we call the `<patch-index>` on the left followed by, state identifiers (not visible in this example), and then the short SHA of the patch and its summary. The following is a description of the structure and the possible state identifiers. git-ps ls result structure: [patch-index] [review status] [commit short sha] [commit summary] "0 rr 788032 Update README with deployment instructions" git-ps ls Review Status: Has NOT requested a review (AKA Empty) rr Requested a review and it's unchanged since requesting rr+ Requested a review and it has changed since requesting p Commit has been published to upstreams mainline ### Submitting a Patch for Review or Integration [git-ps][] also makes submitting a patch for review or integration much easier. Instead of having to do these four steps as discussed in sections above. 1. create a branch based on the current branch's upstream remote branch (e.g. `origin/main`) 2. cherry pick the patch you want reviewed to this new branch 3. push this branch up to the remote (e.g. `origin/this-patches-branch`) 4. create a pull request from this new branch (`origin/this-patches-branch`) into upstream mainline (`origin/main`) [git-ps][] simplifies this down to a single command called **request review** (`git ps rr <patch-index>`). If we wanted to request review or even re-request review of a patch we simply do the following. git ps rr <patch-index> Don't worry if the SHA of a patch changes since your original request for review. [git-ps][] is smart enough to handle this. ### Pull Changes down from Upstream [git-ps][] also helps you pull changes down from upstream so that you can integrate your local patches with the work that has recently been added to upstream. This is as simple as running git ps pull This facilitates fetching the changes from upstream and rebasing your stack of patches on top of them. *Note:* If one of your patches was integrated into upstream, it will simply collapse out of your stack of patches as Git sees that it is a duplicate of a change that is already integrated in upstream. ### Publish a Commit Upstream Once a patch has been approved through the peer review process, you generally want to integrate it into the upstream branch. This can be done by using things like GitHub's green button. However, this often creates useless noise in the Git tree, and also requires additional manual cleanup steps on your part to remove the no longer needed remote branches. So [git-ps][] provides the **publish** command specifically to address publishing the local patch to the upstream branch without creating useless noise in the Git tree and cleaning up the no longer necessary remote branch. This can be done as follows. git ps pub <patch-index> ### See a particular Patch Another feature that [git-ps][] provides is the ability to quickly show the contents of a particular patch. This is as simple as the following. git ps show <patch-index> ### Reorder & Update your Patch Stack In this methodology one of the most common things you do is reorder and update your respective patches while you are iterating on things, and getting them to an acceptable state to request review. As we covered above, interactive rebase `git rebase -i` is an extremely valuable tool that helps facilitate this. Therefore, [git-ps][] simplifies the use of it by using the understanding of the stack of patches concept to automatically fill in the appropriate arguments for the `git rebase -i` command. In turn making an interactive rebase of your stack of patches as simple as git ps rebase ## License `git-ps` is Copyright © 2020 UpTech Works, LLC. It is free software, and may be redistributed under the terms specified in the LICENSE file. ## About <img src="http://upte.ch/img/logo.png" alt="uptech" height="48"> `git-ps` is maintained and funded by [UpTech Works, LLC][uptech], a software design & development agency & consultancy. We love open source software. See [our other projects][community] or [hire us][hire] to design, develop, and grow your product. [Git]: https://git-scm.com [GNU make]: https://www.gnu.org/software/make/ [community]: https://github.com/uptech [hire]: http://upte.ch [uptech]: http://upte.ch [git-ps]: https://github.com/uptech/git-ps</code></pre> Uptech Studio Releasing Standards 2021-05-01T00:00:00+00:00 Here at Uptech Studio</a> given that we use semantic versioning as our standard for versioning</a> we can talk instead about the process of doing a release. Normal release</h2> Cutting a release consists of the following steps. identify the commit in the Git history that is to be the release - generally this is HEAD</code></li> tag the commit with the new release version number, e.g. 1.2.3</code> or v1.2.3</code></li> push tags up to the remote, e.g. git push --tags</code></li> update the CHANGELOG.md</code> if not automatically handled by CI, e.g. git cl full > CHANGELOG.md</code> - Note: if this was an app with build numbers in the version you would use git cl full -p > CHANGELOG.md</code>. commit and push the update CHANGELOG.md</code> commit for that version release</li> </ul> </li> </ul> Avoid Hot-fix release</h2> We avoid "hot-fix" releases if we can. What this means in practice is that if we have a version that is released and we identify an issue with it. Instead of prepping a hot-fix release we just add the fix to the top of the tree and cut a new release including that fix as well as the other changes integrated into mainline. This works because our Branching Strategy</a> is based on Continous Integration and Trunk Based Development which keeps our mainline in a build & releasable state. Hot-fix release</h2> If for some reason we had to do a hot-fix release. There is nothing stopping us from being able to do it. We would have to simply perform the following steps. identify the tagged release that the issue is found on in our Git tree</li> create a new hot-fix branch based on that tagged commit</li> add a commit to that hot-fix branch to resolve the issue</li> tag that commit with the new release version number</li> backport that commit to mainline so the next release will also include it</li> </ul> Automated Releases</h2> We do not setup automated releases for libraries. Apps and Services on the other hand we do generally in the lower environments. But with some projects even in the production environment. In the App case the automated releases are only used for internal release, bumping build numbers, and getting feedback from stakeholders. We generally use MS AppCenter for this. In the Services case the automated releases are fine as build numbers as the interfaces are versioned within the code base. Marketing Releases</h2> When we are doing a public Marketing Release of an application we go through the normal release</a> process with tags and changelog, etc. Uptech Studio Roles 2021-05-01T00:00:00+00:00 Lets first understand exactly what a role is. According to the macOS Dictionary it is "the function or part played by a person or thing". The distinction here is that a role is not a person or their title. It is something a person takes on. For our discussion the role itself is a categorization of work that is to be performed or acted by a person. This means that a person could take on multiple roles potentially. As you can see below the roles of Staff Engineer & Engineering Manager are both played by our Partners. The following are a breakdown of the Engineering roles at Uptech Studio</a>. Developers</li> Tech Lead</li> Staff Engineer (played by Partner)</li> Engineering Manager (played by Partner)</li> </ul> Client Focused</h3> No matter what role(s) a person is playing it is crucial that they are client focused. We are all responsible for understanding the client's business needs and helping to ensure we're always doing what's best for them. For example critically thinking about feature requests and how those impact the user and presenting alternatives that may not be quite as effective, but much less costly in terms of development. This type of thinking and conversation is crucial to have with clients to make sure together we are doing what is best for their business at their current lifecycle stage. We expect everyone within Engineering at Uptech Studio</a> to be able to mentally step back from their technical tasks and critically think about the client's business. Developers</h3> The following are expectations we have of people playing the role of Senior Developer. These are intended to be used as a guiding light for the long term direction developers should be heading as they grow to play a Senior Developer within Uptech Studio</a>. client focused</li> deep experience in specific platform/tech stack with broad experience in other areas (T shaped employee)</li> understanding of engineering processes and practices having a good understanding of how we do engineering here at Uptech Studio</a>, agile processes, git processes, etc.</li> </ul> </li> have sound understanding and usage of application architecture architecting a solution within the scope of a framework</li> architecting a solution outside the scope of a framework</li> identifying framework architectural concepts and boundaries and where to apply framework architecture vs outside scoped architecture</li> </ul> </li> break down problem space ability to understand & collaborate with the Tech Lead around the system architecture, application architecture and further breakdown various libraries, components, services, etc.</li> </ul> </li> drive tasks/components/etc. identifying broken down components, unknowns, etc. and being the driving force to get them prioritized and resolved to aid in accomplishing the primary goal</li> </ul> </li> </ul> Tech Lead</h3> Tech Leads have all the same expectations of a Senior Developer in addition to the following. Tech Leads high level focuses are around Architecture, Foundation, Mentorship, Development, and Process management. more client interactions because they are generally a Squad Representative</li> run tech meetings</li> primary point of contact for engineering questions & interactions</li> working with Product to do planning for Roadmap and tasks</li> architect, problem solve, etc.</li> drive the technical side of achieving the client's goal breaking down the problem space at each of the levels</li> monitoring progress</li> removing road blocks for team mates</li> course correcting</li> </ul> </li> have sound understanding and usage of infrastructure (AWS, terraform, kubernetes)</li> have sound understanding and usage of backend & front-end development</li> </ul> Partners</h3> Due to the current size of Uptech Studio</a> there are a number of roles, Staff Engineer & Engineering Manager, that are strictly played by our Engineering Partners, Drew De Ponte</a> & Claude Ciocan</a>. It is possible that as the company continues to grow we may decide to have other people take on these roles. However, for the moment they are handled by the Engineering Partners. Staff Engineer</h4> Staff Engineers are intended to be a resource shared across the organization to help with standardization, best practices, mentorship, and contribution. Staff Engineers are expected to have all the same expectations of a Tech Lead in addition to the following. shared across the engineering organization</li> advising tech leads and developers</li> advising on architecture</li> advising on Uptech Studio</a> process</li> advising on development process/tooling</li> advising on coding best practices</li> helps on specific tasks</li> </ul> Engineering Manager</h4> The Engineering Manager role is responsible for being direct managers of the Developers & Tech Leads from an HR standpoint. We believe in servant leadership and the idea of trying to understand peoples interests so that we can find their best fit and growth path within Uptech Studio</a>. This involves working to try to align people with projects that would most interest them or aid in their growth in specific areas of interest. We have found that when people are aligned with their interests they tend to do their best work. Uptech Studio Testing Standards 2021-05-01T00:00:00+00:00 We have always been big believers in automated testing and Test Driven Development (TDD</a>). Back in the day we were even big believers in Behavior Driven Development (BDD</a>). Over the many years we have learned that Cucumber</a> and Gherkin</a> were nice ideas but we never really saw huge value and it required additional adoption from Product which was always a hurdle. The concept of emergent design, the process of evolving application architecture from mocked tests, sounds great in theory. But in practice we didn't see any real added value once you have written the non-mocked tests. This is because the mocked tests become brittle noise that isn't worth maintaing and in fact interfere with the process of refactoring functionality. Even if you take the stance of removing these mocked tests, the development team has then taken on the burden of identifying which tests to remove and when to remove them. This is simply more overhead that doesn't provide any real payout. Now there is no reason to throw the baby out with the bath water. There were ideas in BDD</a> that we found to be extremely valuable. For example the concept of applying Outside-In Development as a way of making sure that what is being developed throughout the application architecture is actually necessary. We found this practice aids immensely with not only making sure the right software is built but also with the process of identifying architectural boundaries and doing application architecture. Therefore our current position on testing is a hybrid of the various methodologies we have used over the years. The principles we follow are listed below. Outside-In</li> Test First</li> Small Tests</li> Red, Green, Refactor</li> State over Behavior</li> Unit & Integration</li> Setup in Test</li> Full-Stack Integration within Reason</li> </ul> Outside-In</h2> As described above we value Outside-In Development from BDD</a> as a way of making sure that what is being developed throughout the application architecture is actually necessary. Additionaly we value it as a tool for aiding with architecting an application as well as identifying application architecture boundaries. Which are both extremely useful when writing software in a fashion supporting collaboration. You can see more on Outside-In and how it relates to some of our other processes in our Journy to Small Pull Requests</a> and How we should be using Git</a> posts. Test First</h2> We also strongly abide by the concept of "Test First", the practice of writing a failing test against a nonexistent idealized API and then following the Red, Green, Refactor process to iterate the test & implementation to its final state. The primary reason we value "Test First" so much is because the practice of thinking about what the ideal API would look like and using the first iteration of writing the test as a tool for this has huge positive impacts in terms of the application architecture and its maintainability over time. Beyond that it also benefits in helping to make sure that the tests fully cover the implemented code. If instead you were to for example to write a function first and then go back and write the test after the fact. You would discover over time that it is very easy to miss cases and end up with a test suite that doesn't fully cover all the cases. Small Tests</h2> As a general rule of thumb we stick to one conceptual assertion per test. What this means is that we effectively want to test and verify one thing. If we are testing say a function that returns an object that should have multiple attributes on it in a new state. Then we would have multiple assertions to verify those. But conceptually we are still verifying the resultant state of that one action. There are cases, when doing higher level integration tests especially, where it may make sense to actually do multiple conceptual assertions. However, this is the exception not the rule. Sticking to one conceptual assertion keeps our tests focused in such a manner that when one fails it is focused enough that we don't have to do a ton of debugging to identify what the problem is. Generally the failing test immedietly tells us exactly what the problem is. This of course does not hold true for higher level integration tests, especially the ones that have multiple conceptual assertions. Red, Green, Refactor</h2> Red, Green, Refactor is really just a name for a process that occurs naturally when doing "Test First" TDD</a>. It simply communicates the enforcement of the sequence of the testing process. write a failing test - a.k.a. Red</li> write code that makes the test pass - a.k.a. Green</li> clean up the the implementation that makes the test pass by refactoring it - a.k.a. Refactor</li> </ol> Over the years this naming has been overloaded by being utilized at a micro level to explain the iterative process of doing "Test First" TDD</a>. In this context is more along the lines of the following. write a failing test (in initial state, not final state) - a.k.a. Red</li> write code that makes the test pass - a.k.a. Green</li> evolve the test to test more of what it should, leaving you with a now failing test - a.k.a. Refactor, Red</li> write code that makes the test pass - a.k.a. Green</li> ...</li> ...</li> evolve the test to test to its final state, leaving you with a now failing test - a.k.a. Refactor, Red</li> write code that makes the test pass - a.k.a. Green</li> clean up the the implementation that makes the test pass by refactoring it - a.k.a. Refactor</li> </ol> I don't think this overloaded adaption to this micro usecase is necessarily the cleanest mapping. But it is what a decent portion of the community has starting using it to refer two. So we might as well at least be clear about it. As I stated before these are basically just the naturaly processes that occur when doing "Test First" TDD</a> so we naturally abide by them. State over Behavior</h2> The concept of State over Behavior means that we ideally want to do State based testing whenever possible and only use Behavior testing when explicitly necessary. We hold this stance because State based testing aids with application architecture design and is naturally long living as well as interface abiding. This means that once we have written a State based test it is useful when we go to refector the implementation of a function/method. Behavior tests are not useful and have to be completely reworked when refactoring implementation of a function/method. This position does have implications. For example to support isolation (via stubs/test doubles) in our tests we need to modify how we write our code to make sure we are using Dependency Injection. This is an implication we are more than happy to take on as we have found that classic Dependency Injection has aided massively in our application architectures over the years. Unit & Integration</h2> There are two main categories of tests Unit and Integration. We are proponents of both of them. Unit tests are focused around validating a particular unit of code, generally a function/method. Therefore, they isolate away collaborator objects. An example might look something like the following. stub_shopping_cart = ShoppingCartTestDouble() add_item(item, stub_shopping_cart) assert(shopping_cart.items.include(item))</code></pre> In the above example we replaced the shopping cart by giving the add_item</code> function the stub_shopping_cart</code>. This isolates away any functionality that a real shopping cart whould have had and allows us to strictly test the functionality of just the add_item</code> function. Integration tests are focused around validation of many units of code working together in an integrated fashion. An example might look as follows: data_store = DataStore() real_shopping_cart = ShoppingCart(data_store) add_item(item, real_shopping_cart) assert(shopping_cart.items.include(item))</code></pre> You can see in the above that it is using real objects rather than Test Doubles or Stub objects. The benefit of this is that it can be used to verify functionality of all those different units of code working together which is also important. Especially to make sure that integrated results are what we expect. Therefore we write both Unit and Integration tests. Setup in Test</h2> Setup in Test is the concept of making sure that the setup portion of each test is included directly in the test itself and not in some before each</code> or describe</code> block or some parenting test framework concept that provides initial state setup across multiple tests. You might be thinking to yourself well this concept goes against the DRY principle which we follow in implementation code. Shouldn't we be following it in our tests as well? Technically it doesn't go againts the DRY principle overall. It just goes against DRYing test setup up using framework concepts like before each</code> and describe</code>. The reason that Setup in Test exists as a concept is because extracting test setup into before each</code> or describe</code> blocks or other test framework constructs have massive negative implications when you are coming back to a test later on. They make it very difficult to understand what the exact test setup is for a particular test. You often have to go hunt to try and figure out what test setups are done earlier in the suite and even then there are likely test setup steps that aren't needed in your particular test but exist earlier in the suite. To get the best of both worlds, clarity of test setup and DRYing up test setup we follow the common strategy of extracting test setup into utility functions in the test suite. This enables us to have clear, concise, well named utility functions that we can use in each of the tests that clearly communicate the setup. Even in cases where the naming don't clearly communicate the setup, it is easy enough for us to dive into any setup functions and see what is going. This is superior as there is a formal relationship and we don't have to guess if setup is used or not. Full-Stack Integration within Reason</h2> Full-Stack integration testing is the same concept as integration testing from above. The difference is that it is generally at the highest level of the application and integrates through all the various layers of the application. We are big believers in this for the same reason we are integration testing. However, when you are doing Full-Stack integration testing you often have to deal with things like a database or 3rd party APIs, etc. This is why we have the caveat of "within Reason". Doing true Full-Stack integration testing is very difficult and not practical/pragmatic in most cases. It often makes more sense to stub out 3rd party integrations and do some sort of datbase resetting/transaction management, or maybe an in memory database substitution. In some cases maybe it even makes sense to start a layer lower in the architecture. The point is that given the impracticality of doing true Full-Stack Integration Testing we instead figure out for each of the different projects we work on what we can do "within reason" to get as close to Full-Stack Integration as possible. Uptech Studio Versioning Standards 2021-05-01T00:00:00+00:00 Versioning Libraries</h2> We use Semantic Versioning</a> for our standard of choice for versioning libraries. Versioning APIs</h2> There are few options. URL based versioning, e.g. /api/v1/route</code></li> HTTP Header based versioning use Media types</a>, e.g. application/vnd.github[.version].param[+json]</code> or application/vnd.github.v3+json</code></li> </ul> </li> HTTP Custom Headers versioning (any header beginning with X-</code>), e.g. X-GitHub-Media-Type</code></li> </ul> We prefer and will use HTTP Header based versioning with Media Types as it is the most robust solution that generally also has support built into libraries or frameworks. The HTTP Custom Headers versioning is technically more robust as there are no standards around it. However, it then requires more custom work to do be done for clients interacting with it. The URL based versioning is NOT robust because it requires either duplicating code, or implementing some sort proxying, etc. to deal with situations where you want to create another version of all the routes. It also makes it harder to mix them as well in terms of code management. How we should be using Git 2021-01-30T00:00:00+00:00 Git was created by Linus Torvalds out of a need. At the time the Linux Kernel team was using a proprietary Distributed Source Control Management (DSCM) system. However, due to licensing issues the Linux Kernel team could no longer use this proprietary DSCM system. Therefore, Linus decided to build Git as the DSCM system he always wished they had. You might also know Linus as the creator of Linux. In fact, Linus still manages the Linux Kernel today. As of 2020, the Linux kernel had over 27.8 million lines of code spread across ~66 thousand files from ~21 thousand different contributors. It has continued to be successfully developed, maintained, and extended since it was publicly announced in 1991. It is also worth noting that Git itself, another large, successful open source project, is managed in the same way. So there must be some useful insights around long term software development and maintenance practices we can glean by looking at how Linus and his team use Git for their development and peer review workflows. How the Linux Kernel Team works?</h2> It may sound crazy but it turns out that the Linux Kernel and Git teams use email for both their code review process and their code submission process. They use a concept from the Unix days called a patch. A patch is a simple text file that describes a change to another text file's content. In Unix there exists a tool called diff that takes two files and produces a patch file. People often refer to these patch files as diffs. Another Unix tool called patch takes a patch file and applies it to another file. Back in the day this Unix patch concept is actually how open source development was done. The contributor would make a change to some source code, produce a patch, and email that patch to the maintainers requesting feedback. The contributor would then take the feedback and make the adjustments to the code. Then they would produce a new patch and email it back to the maintainers for further review and possible integration. Fundamentally this is how peer review is done with the Linux Kernel and Git teams today. The primary difference is that they use Git to manage changes locally and then use a number of Git commands to generate a patch or a sequences of patches and email them to the maintainers. They also use a number of Git commands to consume and apply patches from emails. See git apply</code>, git am</code>, git format-patch</code>, git imap-send</code>, and git send-email</code>. In addition the team also requires that patches have certain characteristics. a patch must be small, logical, independent, buildable, & testable</li> a patch must provide a clear description of what changed, why it was changed, and the necessary context of how it is or will be used</li> patches must be submitted individually for review</li> </ul> If we dig a bit further into the Linux Kernel team's process we find that they have integrators responsible for the various subsystems. These integrators receive numerous patches from various contributors and manage them locally as a series of patches (using quilt, another Unix command). This involves reordering, updating and dropping them on a regular basis. Linus then regularly pulls the changes from the various subsystem integrators into the upstream repository's mainline. What did we learn?</h2> From a small dive into the Linux Kernel team's development process we can make three important observations. First, the process imposes a strict set of requirements for each patch before it may be sent for review or submission. Second, the team's process works well over email, a very asynchronous communication mechanism. Finally, they locally use the concept of managing a series of patches while preparing changes for integration into upstream. So what's the Magic?</h2> Is it the Kernel team's avoidance of powerful peer review tools that makes them so successful? Or the fact that they do this all over email? I think not. I believe their success is primarily due to their process. The specific requirements for patches are fundamentally in alignment with the core design principles of Git. They are also structured around the concepts of boundaries of the application architecture. Beyond that, it seems this mental model of locally managing a series of patches as they are prepared to be submitted for review or submission is key. Outside-In</h2> Let's assume for the time being I am correct and we trust that following these rules and this mental model will greatly benefit us. (Later we can explore the actual values we get from these rules and why they are so important.) How can we use this to benefit our own projects? Before we can apply these rules we need to first understand how to break down our code changes into small logical independent chunks that are buildable and testable. To help us with this, let's introduce a new mental model for thinking about application architecture in relation to boundaries, called Outside-In. With Outside-In you begin your approach to code changes by starting from the outermost (UI/UX) layer of the architecture, and drive layer-by-layer down into the architecture until you reach your core dependency, the inner most layer of the relevant architecture; you then reverse direction and work your way out again, wiring the layers together as you go. This method ensures that you are building exactly what you need-nothing more, nothing less. It also helps as a tool to recoognize different layers of abstraction within your application which aids massively in thinking about the overall application architecture. Last, it also drives the interfaces of your abstracitons so you end up with more clearly defined boundaries. Sounds great in theory</h2> But what does this actually look like? Let's say we are tasked with adding the ability to login to our mobile app. If we follow Outside-In this might look something like the following series of steps: add login view (incomplete because depends on a API client that doesn't exist)</li> add login support to API client (incomplete because depends on REST API lib)</li> add lib to facilitate making REST API requests (our core dependency)</li> complete add login support to API client (wiring it up to the lib added in the prev step)</li> complete add login view (wiring it up to the API client login ability added in the prev step)</li> </ol> If we created commits locally for each of these steps our local commit tree would look like the following: * complete add login view (HEAD, main) * complete add login support to API client * add lib to facilitate making REST API requests * WIP: add login support to API client * WIP: add login view * ... (origin/main)</code></pre> This is fine, but these don't look like logically chunked commits. They are also in the incorrect order based on dependencies. Luckily we are working in Git, which provides the interactive rebase (git rebase -i</code>) command which facilitate reordering and squashing commits together, amongst other valuable things. Before we fix these issues, let's just look at what the tree would look like if we reordered it based on dependencies and didn't squash the commits. * WIP: add login view (HEAD, main) * complete add login view * WIP: add login support to API client * complete add login support to API client * add lib to facilitate making REST API requests * ... (origin/main)</code></pre> From this ordering we can start to actually see the relation and the logical chunking. For example the "WIP: add login view" and the "complete add login view" are two commits that are part of the same logical chunk. If we identify these commits and squash them together into their respective logical chunks we would end up with a local Git tree that looks as follows: * add login view (HEAD, main) * add login support to API client * add lib to facilitate making REST API requests * ... (origin/main)</code></pre> The Kernel team describes a series of patches as a "proof of work". Think back to when you were a kid in school and your teacher required you show how you got to a solution for each problem. This concept is effectively the same thing. Each logical patch in the series is a step of you showing how you are solving the problem. This is a another great mental model and is useful for making sure your series of patches make sense. Where are we with these rules?</h2> At this point we have figured out how to locally iterate on commits to get them to a point where they effectively are a series of small, logical, independent, buildable, and testable patches. That leaves us with the following: a patch must provide clear description of what changed, why it was changed, and the necessary context of how it is or will be used</li> patches must be submitted individually for review</li> </ul> Meeting the first of these is simply writing a good commit message. Check! But how do we follow the second requirement to submit patches individually? Submit for Review or Integration</h2> We could, of course, setup a mailing list like the Linux Kernel team and use the same Git commands they do. However, this seems archaic when we have tools like GitHub, Bitbucket, Gitlab, etc. that provide the concept of a Pull Request, as well as code review tools that are much nicer than email. Not to mention the fact that the majority of the development community is already comfortable with these tools due to the unjust popularity of Feature Branch development. So how do we create a pull request of an individual patch once it is ready for review? create a branch based on the current branch's upstream remote branch (e.g. origin/main</code>)</li> cherry pick the patch you want reviewed to this new branch</li> push this branch up to the remote (e.g. origin/this-patches-branch</code>)</li> create a pull request from this new branch (origin/this-patches-branch</code>) into upstream mainline (origin/main</code>)</li> </ol> Now reviewers can review the changes as they normally would. They will have a clear description of what changed, why it was changed, and the necessary context of how it is or will be used, and will be able to easily review the actual code, testing them out, and providing feedback. A little later we will introduce a tool that automates these steps for you, making this quick and easy. So What's the point?</h2> Interestingly, it turns out that structuring your patches (a.k.a. commits) logically around the concepts in your application architecture is a major part of the magic. This specifically aids with understanding concepts, how they are introduced, and how they are modified over time to support higher level concepts and features. This is what largely facilitates long term maintenance of software. This method also facilitates requesting review for individual patches as soon as they are completed. I can't emphasize how important this is. This enables us to basically develop Outside-In but request review Inside-Out. Meaning as soon as the inner most dependency is complete we can request review of that and get feedback all while we are still working on the layers above it. Beyond that these small logical, buildable, testable patches result in a faster mean time to repair, extremely focused and valuable reviews, support for fault isolation, decreased code review time, and promote test reliability. They also facilitate integrating patches sooner, which in turn promotes sharing, refactoring, discovery, and natural evolution of application architecture concepts. Not to mention helping to prevent duplicate efforts and improve the overall release rate. Again these small logical, buildable, testable patches fall in line with Git's principles of design and in turn help to unlock the full powers of Git. Specifically in Git there are certain extremely powerful commands (e.g. git bisect</code>) that only function well if your Git tree is composed of logical, buildable, testable commits. The concept of a series of patches also ends up having a huge benefit of eliminating urgency around having one's pull requests reviewed immediately. This is because even after you have submitted a patch for review, you still have that patch in your series of patches locally and can continue building on it without issue. This significantly aids in reducing interruptions between developers, breaking their focus, and inevitably making things take longer. This can't be obtained using feature branches without huge overhead and it is a key differentiator in supporting asynchronous workflows. How do you make sure people are aware of new pull requests without interrupting them? Don't worry-this is a problem that all pre-review based development methodologies suffer from. To help address this issue we built Pullwalla</a>, a unified pull request manager. Finally, following these rules also results in your Git commits and in turn your Git tree representing "proofs of work", the mental construct the Kernel team uses to vet a series of patches. Turns out these "proofs of work" provide an immense amount of guidance and value in simply understanding a code base's application architecture and the intentions behind the various logical changes. Patch Stack</h2> Given that we have found this development methodology to be unmatched in value, we decided we must formalize it a bit more. We decided to call this the Patch Stack methodology, as it resembles a stack of papers that can be reordered and modified. Achieving the full value of the methodology requires that these rules are followed and that you locally manage your commits as if they are conceptually a stack of patches being reordered and modified in preparation for submitting for review or integration. a patch must be small, logical, independent, buildable, & testable</li> a patch must provide a clear description of what changed, why it was changed, and the necessary context of how it is or will be used</li> patches must be submitted individually for review</li> </ul> In addition to formally defining the rules, we have built git-ps-rs</a>, an OSS Git command line extension to facilitate making this workflow and methodology even easier to use. Getting the Feel for It</h2> To get a better feel for how this methodology works in real life let's walk through our example from before. If you recall we are tasked with adding Login ability to a mobile app. Where do we start?</h3> We take guidance from Outside-In and start at the outer most layer of our application architecture (UI/UX). We first look at the existing UI layer of the application architecture to find out if it provides any support for login. Turns out it doesn't. So we are off to the races, building out a login screen and adding it to the flow of the app. We get to the point where we are attempting to wire up the submit button of the login screen and realize our client API for our backend doesn't support login. So we figure out the ideal interface that should get added to the our client API to facilitate login. We also implement code to use this imaginary interface. This code obviously won't build and isn't complete, but we include it anyways as it acts as a note to ourselves for what the ideal interface should be. Given we are following Outside-In we also recognize that we are at a boundary between application architecture concepts, specifically the UI and the client API. This is our clue that we need a patch to house this logical (UI) piece of the puzzle. So we add a patch on top of our stack by first staging the UI changes, gps add .</code>, and then creating the patch. gps create-patch</code></pre> There is an alias of gps c</code> for short as this operation is so common. When it prompts for the commit message we provide something like the following, making sure to provide a clear description of what changed, why it was changed, and the necessary context of how it is or will be used. WIP: Add login screen So that users will be able to login to their account. This interfaces with the client API to actually perform the login and get the authentication token.</code></pre> We used the convention of prefixing our commit summary with "WIP:" as a reminder to us that this patch is still a Work In Progress. Onto the Client API</h3> Now that we have created our WIP patch for the UI layer. We move onto to the next layer down, the client API. So we extend the client API by adding our ideal interface and starting to implement the functionality behind it, making any necessary adjustments to the interface as we go. Soon we realize that we don't have a library to facilitate interacting with the REST API. We also recognize this as yet another boundary in the application architecture. Therefore, we create another patch on top of our stack by staging the changes, gps add .</code> and then committing. gps create-patch</code></pre> We provide the following message with the patch. WIP: Add BackendClient.login(username:password:) method So that our application has the ability to make the login request to the Backend REST API given the username and password. This is intended to be used by the login screen to help facilitate performing the actual login. It will be interfacing with some REST API to actually make the REST call.</code></pre>What's our Patch Stack look like?</h3> We have added a couple patches to our Patch Stack and are ready to check where we are in our process. To do this we simply list our patch stack using the following. gps list</code></pre> An alias for this command is provided as well, gps ls</code>. It outputs a view of all the patches on our stack. 1 23d7a6 WIP: Add BackendClient.login(username:password:) method 0 af73d2 WIP: Add login screen</code></pre> Things look good and we are reminded that we just added the BackendClient.login()</code> method and it is incomplete because of a missing dependency, the REST API library. Onto the REST API library</h3> So we do some research and find what we think is the best REST API library and add it to our Package.swift</code>, making it available to our app. This is yet another boundary in our application architecture and therefore we create a new logical patch for it by staging the changes, gps add .</code>, and committing. gps create-patch</code></pre> With the following message: Add Siesta, the elegant way to write REST clients So that we can easily and most importantly make elegant REST requests. We intend to use this inside of our BackendClient.login(username:password:) method to elegantly make the login REST API request.</code></pre> It is worth noting that we do not use the "WIP:" prefix in our patch summary here. This is because it is a complete independent logical patch and happens to also be our core dependency in the context of Outside-In. Where is our Stack at?</h3> Let's check out our stack of patches now. gps list</code></pre> It outputs the following. 2 4387d1 Add Siesta, the elegant way to write REST clients 1 23d7a6 WIP: Add BackendClient.login(username:password:) method 0 af73d2 WIP: Add login screen</code></pre>Reorder our Patches by Dependency</h3> Things are looking pretty good but our patches are not ordered correctly based on dependencies. Let's fix this by reordering them using gps rebase</code> and changing them into the following order. pick 4387d1 Add Siesta, the elegant way to write REST clients pick 23d7a6 WIP: Add BackendClient.login(username:password:) method pick af73d2 WIP: Add login screen</code></pre> This order is correctly ordered by dependency, as "Add Siesta..." is the inner most core dependency, followed by "WIP: Add BackendClient.login..." being the layer that directly depends on "Add Siesta...". Following that is the "WIP: Add login screen" patch which depends on "WIP: Add BackendClient.login...". Pull Changes from Upstream</h3> At this point we have a logical, buildable, testable patch that is ready to be submitted for review. However, lets make sure that our patch stack is good with all the other work people have recently published to upstream. We do this as follows. gps pull</code></pre> This takes the stack of patches and essentially lifts them up and replays them on top of the upstream changes. The gps pull</code> ran successfully with no conflicts. So all the patches in our stack are good. 2 af73d2 WIP: Add login screen 1 23d7a6 WIP: Add BackendClient.login(username:password:) method 0 4387d1 Add Siesta, the elegant way to write REST clients</code></pre>Request Review</h3> We now request review of our "Add Siesta..." patch as it is a complete logical patch and we want people to be able to review it while we are working on implementing the rest of our feature. gps request-review 0</code></pre> An alias for this command is provided as well, gps rr <patch-index></code>. For this example lets assume that we are using GitHub</a> and have installed the request_review_post_sync</code> hook that integrates with GitHub</a>. In the case where we initially request review of a patch it creates a pull request for us on GitHub</a>. Getting feedback on this inner most dependency first is important as the architectural decisions around it can impact the work in the layers above. Back to the Client API</h3> One of the benefits of working in a stack of patches is that we can continue building on top of our previous patches even after we have requested review of them. Now that we have the Siesta library available to us, we go back to our BackendClient.login(username:password:)</code> method and finish implementing it using the Siesta library. Once we are done we create a new patch on our stack with the message, "finish BackendClient.login(username:password:)", using gps add</code> & gps create-patch</code>. Note: We don't care so much about this patch message as it is a temporary patch. We just need to know what it does in relation to the other patches in our stack. If we checkout our stack of patches with gps list</code> it looks something like this now. 3 df71da finish BackendClient.login(username:password:) 2 af73d2 WIP: Add login screen 1 23d7a6 WIP: Add BackendClient.login(username:password:) method 0 rr 4387d1 Add Siesta, the elegant way to write REST clients</code></pre>Reorder & Squash</h3> Looks like we have the makings of another logical patch. However, it is made up of two separate patches, the "WIP: Add BackendClient.login..." patch and the "finish BackendClient.login..." patch. Lets reorder these patches and squash them together using gps rebase</code> to turn them into a single logical patch. pick 4387d1 Add Siesta, the elegant way to write REST clients pick 23d7a6 WIP: Add BackendClient.login(username:password:) method s df71da finish BackendClient.login(username:password:) pick af73d2 WIP: Add login screen</code></pre> When we save and quit the editor it will open the squashed patch descriptions up to create the singular patch message. We remove the temporary patch message and remove the "WIP:" prefix from the summary as this is now a complete, buildable, testable, logical patch. If we list our patches with gps list</code> again. Our patch stack now looks as follows. 2 af73d2 WIP: Add login screen 1 23d7a6 Add BackendClient.login(username:password:) method 0 rr+ 4387d1 Add Siesta, the elegant way to write REST clients</code></pre>Review Feedback</h3> We just got a notification that someone reviewed our first patch and requested some changes. Instead of using Siesta as our REST library they want to use Alamofire and have some decent arguments for doing so. So looking at our stack of patches we need to edit the "Add Siesta..." patch to change it to Alamofire. We can accomplish this by using gps rebase</code> and changing the "Add Siesta..." patches rebase command from pick</code> to edit</code>. When we save and quit the editor it will drop us into being able to edit the patch we marked with edit</code>. So we make the necessary changes, removing Siesta from the Package.swift</code> and adding Alamofire. We then stage these changes with gps add .</code> and amend them to the "Add Siesta..." patch using gps amend-patch</code>. This in turn opens our editor with the patch message allowing us to update it. Add Alamofire as a REST client So that we can easily make REST requests. We intend to use this inside of our BackendClient.login(username:password:) method to make the login REST API request.</code></pre> Once we have saved and quit the editor the patch has been updated. However, we still need to complete the rebase which stopped in the middle. To do this we simply run gps rebase --continue</code>. Re-Request Review</h3> If we checkout our patch stack it should look something like this. 2 af73d2 WIP: Add login screen 1 23d7a6 Add BackendClient.login(username:password:) method 0 rr+ 4387d1 Add Alamofire as a REST client</code></pre> The rr+</code> is indicating that there have been changes made to that patch since we last requested review of it. So we need to re-request review so that the reviewer can see the recent changes we have made. We accomplish this by simply running gps request-review 0</code> again. This will update the previously created pull request for this patch so that the reviewer and can re-review it. Rework BackendClient</h3> Now that our core dependency is ready for review again we can focus on the layer above that again. Since our lower layer dependency changed we need to update our "Add BackendClient.login..." patch to use Alamofire instead of Siesta. To do this we simply use gps rebase</code> again and mark the "Add BackendClient.login..." patch for edit</code>. It drops us back to the console waiting for us to edit that patch. We do so by making the necessary changes, staging them with gps add .</code>, and then amending them with gps amend-patch</code>. When prompted to the update the message we simply save and quit the editor because the message is still correct. Then we run gps rebase --continue</code> to have it finish the rebase process. Leaving our patch stack as follows. 2 af73d2 WIP: Add login screen 1 23d7a6 Add BackendClient.login(username:password:) method 0 rr+ 4387d1 Add Alamofire as a REST client</code></pre>Request Review of the 2nd patch</h3> Now that we have our second patch complete. We pull the upstream changes again with gps pull</code> and then request review of our second patch with the following. gps request-review 1</code></pre> Creating a pull request and leaving our stack of patches in the following state. 2 af73d2 WIP: Add login screen 1 rr 23d7a6 Add BackendClient.login(username:password:) method 0 rr+ 4387d1 Add Alamofire as a REST client</code></pre>Approval Arrives</h3> Meanwhile the approval notice has come in for our "Add Alamofire..." patch. So we integrate this patch upstream by running the following. gps integrate 0</code></pre> There is an alias for this command as well, gps int <patch-index></code>. This will push that patch up to the remote branch leaving our stack of patches in the following state. 2 af73d2 WIP: Add login screen 1 rr 23d7a6 Add BackendClient.login(username:password:) method 0 int 4387d1 Add Alamofire as a REST client</code></pre>Pull and Collapse</h3> We decided to pull in any changes from upstream with gps pull</code> making our patch stack now look as follows. 1 af73d2 WIP: Add login screen 0 rr 23d7a6 Add BackendClient.login(username:password:) method</code></pre> You will notice that the "Add Alamofire..." patch has left our patch stack. This is because it has conceptually moved from our stack of patches into its upstream branch. Back to the UI</h3> We have now worked our way back up to the outer most layer of our application architecture, the UI. Here we make changes to wire the submit button of the Login screen up to the BackendClient.login(username:password:)</code> method completing this feature. Since our "WIP: Add login screen" patch is the top most patch. We can simply amend these changes to that patch by staging the changes with gps add .</code> and doing the following: gps amend-patch</code></pre> This opens the patch message in our editor and allows us to remove the "WIP:" prefix from the patch summary. If we list our patch stack with gps list</code> it should look as follows. 1 af73d2 Add login screen 0 rr 23d7a6 Add BackendClient.login(username:password:) method</code></pre> Then we integrate upstream changes with gps pull</code> and request review of our final patch using gps request-review 1</code>. Leaving our patch stack in the following state. 1 rr af73d2 Add login screen 0 rr+ 23d7a6 Add BackendClient.login(username:password:) method</code></pre>Onto the Next</h3> At this point we have requested review of all our patches for this feature and are waiting for feedback. However, we don't just sit idle, waiting for the feedback. We naturally start our next task in the same manner, Outside-In, and adding, reording, and updating patches on top of our stack. You might be saying to yourself, "Hey, but we have this other feature's patches here." You are correct we do. This is fine and in lots of cases it is actually more beneficial to continue building new features on top of the ones you have locally. And if for some reason you find out that this new feature has priority over the one that was just completed. Simply reorder the patches in your stack to represent that. More Acceptance</h3> Over the next couple hours reviewers find time to review the other two patches and they are both approved. We integrate them using gps integrate 0</code> and gps integrate 1</code> changing our patch stack state to the following. 1 int af73d2 Add login screen 0 int 23d7a6 Add BackendClient.login(username:password:) method</code></pre> After integrating these patches we run gps pull</code> to integrate any upstream changes. In this case it includes the two patches we just published. So when we run gps list</code> now we see an empty patch stack. Feature Complete</h3> At this point our feature has been developed, reviewed, and integrated. We are done with this task and can now move on to the next task. It is just Git</h2> Fundamentally git-ps-rs</a> is just a light layer of automation of underlying Git commands. So it is very important to know Git. Especially in relation to this idea of iterating on your patches to a point where they are ready for review. If we just look at interactive rebases</a> in Git we see that there are many things you can do within an interactive rebase. For example you can squash commits, fixup commits, edit commits, reorder commits, reword commit descriptions, and many more. Also there are other Git commands that can aid in this evolution of your patches. For example git add -p</code></a> and git commit --amend</code></a>. We have provided some commands to help with these, gps add -p</code> & gps amend-patch</code>. Beyond that, it is worth understanding how you can use these tools to do things like Split Commits Up</a>. Please take the time to make sure you understand these Git commands and how to use them. It will not only improve your workflow within this methodology but with using Git in general. And remember, this is just Git. If we ever run into situations where we need to step outside this methodology-say, to use a branch-there is nothing preventing us from doing so. In fact git-ps-rs</a> sees any branch with a tracked upstream branch as a patch stack. So you can think of branches as separate stacks of patches. We have found this flexibility to be useful when working with clients that don't follow the Patch Stack methodology. It enables us to still follow the practices of Patch Stack locally despite our client using a different methodology, e.g. Feature Branch development. This enables us to get at least some of the beneficial values of the Patch Stack methodology while still playing nice with others. Pullwalla - A New Era 2021-01-20T00:00:00+00:00 Woot woot! The first couple releases of the year are out 🎉. These releases include major features as well as important bug fixes. iOS</h2> Pullwalla iOS has always helped you discover and examine pull requests you want to review across all your accounts & repositories. This release marks the beginning of a new era where Pullwalla begins to achieve it's true vision as the primary interface for Pull Request Management, regardless of platform or repository strategy. One of the big pieces of the review and approve puzzle that was missing was the new Pull Request Diffs view. With Pullwalla iOS v1.5.0</a> and v1.5.1</a> you can now access it via the "File Changes" button on the Pull Request Details view. Below is a screenshot in it's beautiful dark mode theme. This view presents all of the diffs associated with the given pull request and their associated hunks in horizontally scrollable panes. Letting you review every last change. The Pull Request Diffs view also looks great in light mode. Bug Fixes</h2> In addition to bringing Pullwalla iOS into a state where it completes the natural chain of discover, examine, review, discuss, and approve, these releases also include a number of critical bug fixes to further stabilize support for Bitbucket Server in both Pullwalla iOS and macOS. If you want to see the nitty gritty details, check out the Pullwalla iOS Changelog</a> and macOS Changelog</a>. Developer Notes</h3> In this new era we are continuing our work to add support for other platforms (e.g. GitHub Enterprise), extend functionality to facilitate the peer review process, and bridge the gap between Pullwalla iOS & macOS with the use of shared SwiftUI</a> components, like our new Pull Request Diffs view. Thoughts or suggestions?</h3> As always, we’d love to hear from you. Cheers, The Pullwalla Team git add -p won't split 2021-01-19T00:00:00+00:00 It is amazing how many developers I run into that don't seem to either know about git add -p</code> (a.k.a. git add patch), and ever fewer that know what to do when git add -p</code> won't let you split your hunks up any further. Given how much I use this command in my workflow to facilitate creating logically chunked, buildable commits I figured I might as well make a post just touching on it. Yes, this is something that less keyboard driven developers often ignore as they use a GUI (ex: Sublime Merge</a>) to stage things at a smaller granularity. To be completely honest I use Sublime Merge</a> to stage changes myself sometimes. However, there are a large number of times where I am already working on the command line with Git</a> and therefore it is quicker and easier just to stay on the keyboard and use git add -p</code>. So lets break things down a bit. git add</h2> git add</code> is a command that allows you to take local changes and move them to the staging area in preparation for making a commit. The idea is that you move all the changes you want in your commit into the staging area and then turn the changes in the staging area into your commit. git add -p</h2> When git add</code> is executed with the -p</code> option it instructs git add</code> to allow you to select a portion of the changes in a local file for staging rather than the entire set of changes in that local file. Lets look at an example to see exactly how this works. For sake of discussion lets say we have the following local, unstaged changes. ❯ git diff diff --git a/README b/README index d325bcb..f8f8d46 100644 --- a/README +++ b/README @@ -5,3 +5,9 @@ Subtitle of README Description of the README First paragraph of the README + +Second paragraph + +Third paragraph + +Fourth paragraph</code></pre> Given that we are following best practices because we want all the awesome Git commands to be able to work on our repo and we want our Git history to be useful we decide we want to create three separate commits, one to add each of the paragraphs in the diff. To do this we first start out by running the following: git add -p README</code></pre> This prompts us with: ❯ git add -p README diff --git a/README b/README index d325bcb..f8f8d46 100644 --- a/README +++ b/README @@ -5,3 +5,9 @@ Subtitle of README Description of the README First paragraph of the README + +Second paragraph + +Third paragraph + +Fourth paragraph (1/1) Stage this hunk [y,n,q,a,d,e,?]?</code></pre>Oh No! Where is s?</h2> Most of the time when you run git add -p</code> it will present s</code> as one of the choices in the output. s</code> stands for split and tells git add -p</code> to split the currently presented hunk into smaller hunks and then ask you if you want to stage the smaller hunks. But in this case and generally when you get your hunks small enough it won't split them any further. At this point you have use one of the other options it provides. Woot woot, e here we come!</h2> The e</code> option stands for edit and it puts us into manual hunk edit mode in our editor. This looks like the following. # Manual hunk edit mode -- see bottom for a quick guide. @@ -5,3 +5,9 @@ Subtitle of README Description of the README First paragraph of the README + +Second paragraph + +Third paragraph + +Fourth paragraph # --- # To remove '-' lines, make them ' ' lines (context). # To remove '+' lines, delete them. # Lines starting with # will be removed. # # If the patch applies cleanly, the edited hunk will immediately be # marked for staging. # If it does not apply cleanly, you will be given an opportunity to # edit again. If all lines of the hunk are removed, then the edit is # aborted and the hunk is left unchanged.</code></pre> As we can see from the comments we can simply delete the '+' lines we don't want in this hunk. So in this case I edit the hunk to look as follows and save and quit the editor. @@ -5,3 +5,9 @@ Subtitle of README Description of the README First paragraph of the README + +Second paragraph</code></pre> If we run git status</code> we can see it resulted in part of the changes be staged and the other part not. Lets verify the parts are as we expect. ❯ git diff --staged diff --git a/README b/README index d325bcb..8f6f97e 100644 --- a/README +++ b/README @@ -5,3 +5,5 @@ Subtitle of README Description of the README First paragraph of the README + +Second paragraph</code></pre> The staged changes are correct. ❯ git diff diff --git a/README b/README index 8f6f97e..f8f8d46 100644 --- a/README +++ b/README @@ -7,3 +7,7 @@ Description of the README First paragraph of the README Second paragraph + +Third paragraph + +Fourth paragraph</code></pre> And the unstaged changes are correct. So now we can make our first commit, git commit -m "first commit"</code>. From here we can simply rinse and repeat the process to end up with isolated logically chunked commits. Git - Splitting Commits 2021-01-19T00:00:00+00:00 When you first discover the importance of logically structured Git</a> commits, which is by the way a core fundamental expectation & design characteristic of Git</a> and how it is intended to be used. You are probably still following your bad habits and creating commits that aren't logical commits. This naturally leads to the question, "How do I split a commit up into multiple commits?" Below I present a simple contrived example so that you can learn the mechanics and process of doing this, as the mechanics & process don't change. The other topic related to this which this post does not cover is the process of taking code and splitting it up into logical chunks. This generally takes an understanding of the application architecture involved & the dependency relationship between the various elements. So lets get to it. TLDR</h2> For those who just want a quick reminder reference here is the TLDR. For those who need a bit more context and detail through the walk through read the sections below. git rebase -i</code> - do an interactive rebase & mark the commit you want to split for edit</code>, it will drop you out into the shell at that commit</li> git reset HEAD^</code> - soft reset the commit</li> git add -p</code> - stage just the parts you want in the first commit</li> git commit</code> - commit the first commit</li> git add -p</code> - stage just the parts you want in the next commit</li> git commit</code> - commit the next commit</li> git rebase --continue</code> - continue the rebase to play the other commits on top of the new commits you created</li> </ul> Initial State</h2> For this example lets assume that we have a Git tree that has the following commits. 49d9eb4 - (HEAD -> main) Add first paragraph of the README d3e4eee - Add Subtitle & Description of README 56fa164 - Add title of README 8da3dd9 - (base) Initial commit</code></pre> As we can see from the first commit on base</code>, 8da3dd9 - (base) Initial commit</code>, it adds the title to the README. ❯ git show 56fa164 commit 56fa1649afa886c309ce8e83c5518b48c8600b6f Author: Andrew De Ponte <cyphactor@gmail.com> Date: Tue Jan 19 23:48:13 2021 -0500 Add title of README diff --git a/README b/README index e79c5e8..4b3b7a8 100644 --- a/README +++ b/README @@ -1 +1,2 @@ initial +Title of README</code></pre> The second commit, d3e4eee - Add Subtitle & Description of README</code>, adds the subtitle and the description as seen in the diff below. ❯ git show d3e4eee commit d3e4eeec96f9c3cd3eadeb4d95256a6326485ed4 Author: Andrew De Ponte <cyphactor@gmail.com> Date: Tue Jan 19 23:48:52 2021 -0500 Add Subtitle & Description of README diff --git a/README b/README index 4b3b7a8..cf3aba6 100644 --- a/README +++ b/README @@ -1,2 +1,5 @@ initial Title of README +Subtitle of README + +Description of the README</code></pre> The third commit, 49d9eb4 - (HEAD -> main) Add first paragraph of the README</code> adds the first paragraph in the diff below. ❯ git show 49d9eb4 commit 49d9eb41bd3a0031f537b3865f6c90329f173bc4 (HEAD -> main) Author: Andrew De Ponte <cyphactor@gmail.com> Date: Tue Jan 19 23:49:51 2021 -0500 Add first paragraph of the README diff --git a/README b/README index cf3aba6..d325bcb 100644 --- a/README +++ b/README @@ -3,3 +3,5 @@ Title of README Subtitle of README Description of the README + +First paragraph of the README</code></pre>Non-Logically Structured Commit</h2> Looking at the commit summaries & the diffs themselves we can see that the second commit, d3e4eee - Add Subtitle & Description of README</code> is actually doing two logical things. First it is adding the subtitle. Secondly it is adding the description to the README. ❯ git show d3e4eee commit d3e4eeec96f9c3cd3eadeb4d95256a6326485ed4 Author: Andrew De Ponte <cyphactor@gmail.com> Date: Tue Jan 19 23:48:52 2021 -0500 Add Subtitle & Description of README diff --git a/README b/README index 4b3b7a8..cf3aba6 100644 --- a/README +++ b/README @@ -1,2 +1,5 @@ initial Title of README +Subtitle of README + +Description of the README</code></pre> Instead of the above what we really wanted to have was one commit that adds the subtitle and a separate commit that adds the description as two isolated logical chunks. Edit Mode</h3> To accomplish this we need to utilize an interactive rebase to enter "edit" mode in the correct place in the Git</a> history. In this particular case we want to rebase the 3 commits we looked at onto base</code>. git rebase -i base</code></pre> This will bring up the following in your editor. pick 56fa164 Add title of README pick d3e4eee Add Subtitle & Description of README pick 49d9eb4 Add first paragraph of the README</code></pre> In the interactive rebase buffer we can change the action for the middle commit to edit</code> so it as follows. pick 56fa164 Add title of README edit d3e4eee Add Subtitle & Description of README pick 49d9eb4 Add first paragraph of the README</code></pre> When you save & quit the editor it will run the specified interactive rebase commands. In this case pick (meaning keep) the first commit and then stop on the second commit allowing for editing because we specified, edit</code>. When it does this will drop you back to the console with a message similar to the following: Stopped at d3e4eee... Add Subtitle & Description of README You can amend the commit now, with git commit --amend '-S' Once you are satisfied with your changes, run git rebase --continue</code></pre>Split the Commit</h3> We want to split the changes currently held in this commit into multiple commits. To do this we need to reset the commit that we are currently on and then partially stage the changes back commit, then stage the other part and commit, and then continue the rebase. So first we have to reset just the commit that we are checked out on and we want to do a soft reset. So we do the following: git reset HEAD^</code></pre> Now if we run git diff</code> to see the now local ustagged changes we see the following. ❯ git di diff --git a/README b/README index 4b3b7a8..cf3aba6 100644 --- a/README +++ b/README @@ -1,2 +1,5 @@ initial Title of README +Subtitle of README + +Description of the README</code></pre> As we can see we now have the changes that add both the subtitle & the description locally. So we first want to stage a commit with just the Subtitle of README</code> portion. To do this we need to use git add -p README</code> to do a partial stage of the README files changes to just stage the Subtitle of README</code> portion. See my post, git add patch won't split</a> for details on how to accomplish this. + +Description of the README</code></pre> If you run git status</code> to check on things at this point it should look like this. ❯ git status interactive rebase in progress; onto 8da3dd9 Last commands done (2 commands done): pick 56fa164 Add title of README edit d3e4eee Add Subtitle & Description of README Next command to do (1 remaining command): pick 49d9eb4 Add first paragraph of the README (use "git rebase --edit-todo" to view and edit) You are currently splitting a commit while rebasing branch 'main' on '8da3dd9'. (Once your working directory is clean, run "git rebase --continue") Changes to be committed: (use "git restore --staged <file>..." to unstage) modified: README Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: README</code></pre> If we specifically check out the staged changes with git diff --staged</code> we get the following. ❯ git diff --staged diff --git a/README b/README index 4b3b7a8..70e7d0a 100644 --- a/README +++ b/README @@ -1,2 +1,3 @@ initial Title of README +Subtitle of README</code></pre> This is exactly what we wanted. But lets make sure the unstaged changes also represent what we want by running git diff</code>. ❯ git diff diff --git a/README b/README index 70e7d0a..cf3aba6 100644 --- a/README +++ b/README @@ -1,3 +1,5 @@ initial Title of README Subtitle of README + +Description of the README</code></pre> Yep looks like they do. So now we just need create the first of the two commits that will replace the commit we marked for edit. This can be done as follows. git commit -m "Add Subtitle to README"</code></pre> From there we can stage the rest of the changes with git add README</code> and create the second commit with the following. git commit -m "Add Description to README"</code></pre>Finish the Rebase</h3> Now that the changes have been split up into separate commits like we wanted we now need to instruct it to finish the interactive rebase that we started with the edit. This is done as follows. git rebase --continue</code></pre> Once it is complete if we checkout our tree it will look as follows. fe12bd6 - (HEAD -> main) Add first paragraph of the README 3566dec - Add Description to README f676187 - Add Subtitle to README 56fa164 - Add title of README 8da3dd9 - (base) Initial commit</code></pre> Exactly what we wanted! Understanding zsh Startup & Configuration 2021-01-18T00:00:00+00:00 This is a little diagram I put together years ago when macOS El Capitan came out and I ran into issues with how it was impacting my PATH</code> environment variable. It ends up that every so often I need to reference it. Today was one of those days. However, I couldn't remember where I put it. The first place I looked was my blog. But it wasn't there. Therefore once I found it it I decided it was finally time to put it up on my blog. I hope it ends up being as useful for you as it has been for me over the years. Note: This diagram was built with a tool that I very much enjoy using when it makes sense, Monodraw</a>. Git Changelog v3.3.2 2021-01-11T00:00:00+00:00 I just released Git Changelog</a> v3.3.2. To update simply run the following: brew update && brew upgrade git-cl</code></pre>What's Changed?</h2> Support any Branch</h3> It used to be that prior to this release that it would only work on the master</code> branch. However with this release it now functions properly when currently checked out on any branch. It walks the Git tree from where you are currently checked out on and builds the history of the changelog. So you generally probably want to be checked out on your mainline branch when running it. Availability</h2> This release of [Git Patch Stack][] is available via our Homebrew tap for both macOS Big Sur & macOS Catalina. See the README</a> for details this method and others. Git Patch Stack v0.5.0 2020-12-10T00:00:00+00:00 I just released Git Patch Stack</a> v0.5.0. To update simply run the following: brew update && brew upgrade git-ps</code></pre>What's Changed?</h2> Multiple Stacks of Patches?</h3> This release makes it so that Git Patch Stack</a> now works with any branch that has an upstream branch, instead of having the patch stack be limited to the previously hard coded master</code> & origin/master</code> upstream branch. Other Mainlines</h4> This is useful for making Git Patch Stack</a> work with repositories that have other mainlines, e.g. main</code> & origin/main</code>. Stuck in Feature Branches</h4> It is also useful for managing branches that have upstreams in general. Lets say you are working with a client that doesn’t allow for individual patch pull requests but instead wants a feature branch with a series of logically chunked commits (a.k.a. patches, 😜). Git Patch Stack</a> can now be used to manage the patch stack within that feature branch. Removed Explicit Base Branch</h3> This release removes the explicit base branch option from the rr</code> and pub</code> subcommands because I don't think it is needed now that Git Patch Stack</a> has dynamic upstream branch detection. Also, the explicit base branch option was confusing as it existed. If we find the need in the future to facilitate cross-upstream rr</code>s or pub</code>s we will add it back in a better way. Availability</h2> This release of Git Patch Stack</a> is available via our Homebrew tap for both macOS Big Sur & macOS Catalina. See the README</a> for details this method and others. Pullwalla - Big Sur & The Future 2020-11-18T00:00:00+00:00 Hello! I wanted to clue you into some details around a few recent releases. macOS</h2> Pullwalla macOS v2.6.0</a> & v2.6.1</a> both address the following items. macOS Big Sur Support</h4> As I am sure you already know. macOS Big Sur was officially released. With it comes a completely new style that makes macOS Apps feel a little more like iPad apps. Which Pullwalla macOS</a> now supports. Missing Pull Requests</h4> One of the ongoing challenges that we have had all along is the lack of documentation across the various platforms we support. This results in JSON decoding failures on our end, in turn causing a PR or two to not show up every once and a blue moon. We are constantly addressing these issues as we discover them to make sure you never miss a PR. These releases are no different. Combined we addressed five such issues making sure Pullwalla never misses a PR. iOS</h2> Pullwalla iOS v1.3.2</a> & v1.3.1</a> were released addressing the same missing pull requests issues as above in the macOS release. Bridging the Gap (SwiftUI)</h3> These Pullwalla releases on both macOS & iOS mark the beginning of a new age where we have started building cross-platform UI components using SwiftUI</a>. These releases now use a shared SwiftUI View between both Pullwalla macOS & iOS for adding a Bitbucket Server Account. This is just the beginning as all the new features & UIs we are building are being done with SwiftUI. Also any views that we have to adjust or rework are being done in SwiftUI. This will help us bring the same set of features to both Pullwalla macOS & iOS. Thoughts or suggestions?</h3> As always, we’d love to hear from you. Cheers, The Pullwalla team Keep a Changelog without Conflicts 2020-05-11T00:00:00+00:00 For years, like many others, we have maintained CHANGELOG.md</code> files inside our Git</a> repositories for our apps and libraries. We have followed the standards outlined at keepachangelog.com</a> and we have appreciated the benefits it has provided us for preparing releases and for maintaining a clear record of changes included in each release while still being targeted at the appropriate audience. However, these benefits didn't come without drawbacks. The biggest being that while working on a repo with multiple people or developing locally using a patch stack workflow you continually run into Git</a> conflicts solely with the CHANGELOG.md</code> file. These conflicts aren't meaningful in anyway. They are just an impedance to our development flow and process, both as a team and as individual contributors. Well a few weeks ago Anthony Castelli</a> and I</a> decided it was time to remove this impedance not only for our local workflows but for the development workflow the team. We started out by analyzing the characteristics of the CHANGELOG.md</code> approach that we valued and the characteristics that were causing the issues. The biggest turned out to be one and the same, the tight coupling of the source code change to the CHANGELOG.md</code> entry. This tight coupling is what made it easy to prep releases of libs/apps because the developer that made the change, the person with context, also created the associated CHANGELOG.md</code> entry. However, this was also the practice that was triggering the useless conflicts in the CHANGELOG.md</code>. So we hashed this out and eventually came to the realization that we could have the tight coupling of the Changelog entry and the code while avoiding the conflicts all together. We just had to lift the Changelog entry out to the Git</a> commit message. After contemplating implementation approaches and realizing at the core it would be similar to Git Patch Stack</a> we immediately jumped into building a proof of concept Git</a> command line tool extension to facilitate building a Changelog from entries in the Git</a> commits. Over the last few weeks we have been using it in our workflows and iterating on it to make sure it meets everyones needs. This work has resulted in our latest open source project, Git Changelog</a>. Below is a ~15 min video where I introduce the concepts, values, the git-cl</code> tool, as well as it's best practices. </iframe> </div> Pullwalla - Keepin' it Going 2020-03-18T00:00:00+00:00 Hello! You're receiving this email because you subscribed to updates for Pullwalla. Your unified pull request manager. Where are we at?</h4> We are happy to share with you that since the last email update we have made some very significant changes. These are all available now in the releases in the App Stores. Now on iOS, iPad & macOS</h5> One of these changes is that we have ported Pullwalla macOS over to work on iOS and iPad. Now you can enjoy managing pull requests on your Apple devices whether you are on the couch with your iPad and just need to make a quick response or at your computer and doing a deep review. Bitbucket</h5> We have also added support to manage pull requests from Bitbucket Cloud and are in the process of adding support for Bitbucket Server & Data Center. This enables you to manage pull requests across all your GitHub & Bitbucket accounts whether you are a consultant with client repos spread across multiple accounts or just a dev on a team that uses Bitbucket for hosting their library, service, and app repositories. Diff Stats Feature</h5> Let's say you are leaving for lunch in 15 mins and want to jump on a quick PR. How do you find a bite-sized PR to review without digging through repos and wasting those precious minutes? Diff Stats is a new addition to the Pull Request Characteristics that solves this problem by facilitating visual scanning of PR size while also exposing details of additions, removals, and changes when you want them. If there is a PR that has 1000 additions and 1 removal, the PR will show a large green segment for the additions and a small red segment for the removals. Combined these provide you with a quick visual indication of size. Improvements & Bug Fixes</h5> Beyond the above feature additions we have made numerous improvements and bug fixes. The following are some highlights. showing window when dock icon is clicked on macOS</li> notifications for new/updated pull requests for Pro users</li> a compact view mode that limits pull request description body to only one line of text</li> </ul> To get a full breakdown checkout the Pullwalla macOS Change Log</a> or Pullwalla iOS Change Log</a>. Where to next?</h4> At a high level next steps are as follow to start. add Shared Subscriptions across Devices</li> add iCloud Sync of Accounts</li> add Enterprise Integration with both GitHub and Bitbucket</li> </ul> Beyond that we are also working on implementing PR interactions, Details View, PR Comments, Approve/Reject, etc. To keep tabs on where we are going you can always check out our public Roadmap</a>. Thoughts or suggestions?</h4> As always, we’d love to hear from you. Keepin' it Going</h4> Our thoughts and prayers are with all those dealing with COVID-19. With the onslaught of the virus, we have been lucky enough to be a company that is able to shift to fully remote work and stay productive. To help us remain connected with our team, we built TeamSmash</a>, a visual presence tool for remote teams that integrates with Slack. We find it very helpful, so we wanted to share in case it could help your team with the adjustment to remote work. It is a completely free app. Stay safe, stay positive, and keep it going! The Pullwalla team Constraid v7.1.0 2020-01-09T00:00:00+00:00 Constraid</a> v7.1.0 is available and includes a new API. API Layers</h2> In this release we provide a higher level API based on the Proxy pattern</a> to make things nicer. Constraid</a> effectively provides two public API layers now. high level Proxy pattern</a> based API</li> low level function API</li> </ol> Don't worry this new API layer is strictly a convenience API layer. So this release is 100% backwards compatible and all the low level function API will continue to work. In fact the new convenience API layer is actually built on top of the lower level function API. Syntactic Sugar & Chaining</h2> This new API layer is all about providing a nicer, more convenient API. It does so by utilizing the Proxy pattern</a> to support chaining. In turn providing a much nicer API. The original API layer is effectively a set of functions that create and return Autolayout Constraint Collections which can be combined together to make larger constraint collections. An example of this is as follows: (flush(childView, withVerticalEdgesOf: parentView) + center(childView, verticallyWithin: parentView)).activate()</code></pre> Well, dealing with the +</code> operator and the parens is no longer necessary. As the above can now be implemented using the Proxy pattern</a> API as follows: childView.constraid .flush(withVerticalEdgesOf: parentView) .center(verticallyWithin: parentView) .activate()</code></pre> I hope you can agree this is an easier to read API style and it still provides all the important features and functionality of the lower level API. But don't worry. If we somehow accidentally missed something you can always fall back to using the lower level function API we all used before. Pullwalla is now in the App Store 2019-11-18T00:00:00+00:00 Pra is now Pullwalla!</h2> The app that began as a command line tool named Pra has made the next step in its journey with a new name and new features. The goal remains the same: Pullwalla is designed to be a single, unified interface that allows you to easily manage all the pull requests you care about. New Name and Logo</h2> The new name reflects a change in focus from a minimal, text-based list of pull requests to a full-featured, modern client that is both informative and a pleasure to use. The roots of the app haven’t changed—it aims to be simple, helpful, and unobtrusive. Pullwalla, just like its namesake wallaby, takes advantage of its diminutive size to remain fast, nimble, and quiet. New Features</h2> This release features more than just a makeover. We have added characteristics to help you quickly identify the state of your pull requests. New quick filters help you quickly drill down to exactly what you need. Bugs have been squashed. And that’s not all—plenty more features are in the works. New Website</h2> A new name deserves a fresh website</a>. We have created a landing page to highlight features. A support page</a> with frequently asked questions will help you get started quickly. A change log</a> lets you know what we’ve been working on in the past, while the new roadmap</a> allows us to highlight what we’re working on now and what we have planned for the future. More to come...</h2> We’re excited about all the changes we have in store! We hope you find Pullwalla as useful and pleasurable as we do. Please drop us a line</a> if you have any questions, comments, or suggestions. Constraid v7.0.0 2019-10-08T00:00:00+00:00 I just released v7.0.0 of Constraid</a>. This release is small in terms of features however is quite significant. In this release we now provide support for Swift Package Manager</a> (SPM). This is a huge improvement with the recent support for SPM in Xcode. It already seems to be a superior experience to CocoaPods or even Carthage with a few more edge cases that need to be ironed out. But, hopefully those edge cases will be resolved in the near future. Constraid v6.0.0 2019-09-12T00:00:00+00:00 I just released v6.0.0 of Constraid</a>, your favorite iOS & macOS Auto Layout framework, 😉. This release is a major version bump because I included a breaking change, though relatively minor. I strictly follow Semantic Versioning</a>. @discardableResult</h3> In this release I removed the @discardableResult</code> declaration attribute which is a breaking change. I did this because I came to the realization that ever since we removed the automatic constraint activation to better support Auto Layout animations we haven't needed the @discardableResult</code> attribute. In fact when I was pairing with one of our engineers and he forgot to activate a returned constraint collection, I realized it was actually hindering users. Therefore, I removed the @discardableResult</code> declaration attributes. This now causes the compiler to remind you to do something with the returned constraint collection from any of the functions. Hopefully, this will help keep people from running into, "Oh crap, I forgot to activate the constaints". Having the compiler require you to do something with the results shouldn't be a problem as you will either be. activating the constraint collection immediately, ex: Constraid.flush(viewA, withEdgesOf: viewB).activate()</code></li> storing constraint collection for later use</li> using the constraint collection in an expression</li> </ol> All of which satisify the compilers requirement. Relative Position to Center</h3> The other item included in Constraid</a> v6.0.0 is the addition of the following methods: follow(theCenterOf:with:)</code></li> precede(theCenterOf:with:)</code></li> set(viewA, aboveTheCenterOf:)</code></li> set(viewA, beloweTheCenterOf:)</code></li> </ul> Lets take the first two and look at an example of where they would come in useful. To, start lets look at the Bear</a> app's subscriptions view. If we imagine a dashed line going down the middle of the view you can see what elements are naturally related to the center of the view. In this particular case both the Terms of Service and Privacy Policy links at the bottom of the view are relative to the center. To replicate the layout of the Terms of Service and Privacy Policy buttons using Constraid we would do the following: Constraid.follow(theCenterOf: view, with: privacyPolicyButton, by: 16).activate() Constraid.flush(privacyPolicyButton, withBottomEdgeOf: view, insetBy: 16).activate() Constraid.precede(theCenterOf: view, with: termsOfServiceButton, by: 16).activate() Constraid.flush(termsOfServiceButton, withBottomEdgeOf: view, insetBy: 16).activate()</code></pre> From here you can see that you could handle the subscribe buttons in a similar fashion. Or, if you wanted to do something around the vertical center you can do so using set(viewA, aboveTheCenterOf:)</code> or set(viewA, belowTheCenterOf:)</code>. That about covers it. If you found this valuable or interesting please subscribe to my mailing list using the form below. Swift Failable Decodable 2019-08-30T00:00:00+00:00 Problem</h2> First things first. Lets understand the problem. To do this lets look at situation based on the real world issue that drove me to tackle this. Recently while working on Pra</a> I ran into a situation where it was fetching pull requests from GitHub and then failing to decode the JSON payload response containing the collection of pull requests, resulting in getting back zero pull requests. This is an unacceptable experience. The decoding was implemented in the default fashion as follows: public struct PullRequestSearchResponseNode: Decodable { public let author: PullRequestSearchResponseAccount? public let title: String public let body: String public let closedAt: Date public let permalink: URL ... } public struct PullRequestSearchResponseDataSearch: Decodable { public let nodes: [PullRequestSearchResponseNode] public let pageInfo: PullRequestSearchResponsePageInfo }</code></pre> After some investigation I came to understand that it was failing due to not being able to properly decode one specific pull request because of effectively a type miss-match. For simplicity sake lets say the miss-match is happening because the PullRequestSearchResponseNode.closedAt</code> property has a value in some pull request payloads but is null</code> in others. This is a problem because our struct</code> that we are trying to decode into states that it is expecting closedAt</code> to always have a value. Optionals</h2> The correct, easiest, and most direct answer in the case of closedAt</code> is to adjust the PullRequestSearchResponseNode</code> struct so that the closedAt</code> property is an Optional</code>. public struct PullRequestSearchResponseNode: Decodable { public let author: PullRequestSearchResponseAccount? public let title: String public let body: String public let closedAt: Date? public let permalink: URL ... }</code></pre>The Catch</h2> Using an Optional</code> is great. It fixes the problem and Pra</a> is able to successfully decode all the pull requests again. However, there is one major catch. Sadly as much as we feel we know the specifications for a JSON payload it is likely we will miss something or maybe the JSON payload will change over time. Given this will inevitably happen we can't have that breaking our applications. Possible Paths</h2> So we need to come up with some way of handling the fact that we will miss something in the specification or that the specification will change. All Properties Optional</h3> One approach, not that I recommend it, is to make every property on the decodable struct an Optional</code>. This would address a chunk of possible issues in decoding but not all possible issues. Also, it has the massive downside of then forcing the rest of the app to deal with both Optional</code> paths everywhere in the code. Even if it is a property that will always be present. Optional Wrapper</h3> Another approach is to create a generic struct around T</code> that wraps an option property of type T</code> and provides decoding implementation. This could look something like the following: public struct Failable<T: Decodable>: Decodable { let wrappedValue:T? init(from decoder: Decoder) throws { let container = try decoder.singleValueContainer() self.wrappedValue = try? container.decode(T.self) } }</code></pre> We would use this as follows: public struct PullRequestSearchResponseDataSearch: Decodable { public let nodes: [Failable<PullRequestSearchResponseNode>] public let pageInfo: PullRequestSearchResponsePageInfo }</code></pre> This is nice as it effectively makes it so that we can make decoding failable at any of the various object boundaries. In the scenarios where decoding fails for some reason our wrappedValue</code> would simply be null</code>. This is definitely a step in the right direction. However, it has one major drawback. We have no idea why the decoding failed. Result based Failable Decodable</h3> Enter the Result</code> type. This is exactly it's purpose, to manage optionality while also maintaining an associated error. So, instead of building a wrapper struct we can simply extend Result</code> to implement the Decodable</code> protocol and implement the constructor to handle appropriately assigning the success</code> state in successful decodes, and assigning the failure</code> state when decoding fails. This looks as follows: extension Result: Decodable where Success: Decodable, Failure == DecodingError { public init(from decoder: Decoder) throws { do { let container = try decoder.singleValueContainer() self = .success(try container.decode(Success.self)) } catch let err as Failure { self = .failure(err) } } }</code></pre> This is used as follows: public struct PullRequestSearchResponseDataSearch: Decodable { public let nodes: [Result<PullRequestSearchResponseNode, DecodingError>] public let pageInfo: PullRequestSearchResponsePageInfo }</code></pre> This is great as we get the same ability to make objects failable decodables at any of the object boundaries but we also maintain all of the decoding errors to let us know why decoding failed for a given node. Conclusion</h2> If you are dealing with this situation you probably want to use the Result based Failable Decodable approach. I used this technique to solve the problems described in the Pra v1.4.0 Announcement</a>. It facilitated collecting the decode failures and building the debug report so users can send them back to us. Pra v1.4.0 2019-08-21T00:00:00+00:00 Version v1.4.0 of Pra just made it to the App Store. This release was focused on reliability and robustness. The last release was focused more heavily on new features. If you missed that checkout the Pra v1.3.0</a> post. Reliability</h3> After the v1.3.0 release</a> I quickly got feedback that there were some reliability issues. First there were reports of an infinite loading spinner. After some investigation we discovered it was a side effect of JSON decoding failing in the scenario where a pull request has a requested reviewer that is a GitHub team. So, we resolved both of these bugs in this release. Robustness</h3> We also realized while digging into the above mentioned reliability issues that we had the following robustness issues. The existing logging wasn’t generally valuable</li> The decoding of the fetched pull requests would fail if any of the decoding failed. So, if it couldn’t decode 1 out of 20 pull requests the user would still get 0 pull requests back.</li> When JSON decoding failures happened we had no way of easily finding out or getting useful information that would help us resolve them in the future.</li> </ul> To address these robustness issues I made tho following changes: Migrated logging from mix of NSLog</code> and print</code> calls over to the newer standard OSLog</code> interface.</li> Eliminated useless logging, cleaned up and categorized the logging I kept so that users would have a much easier time finding valuable logs.</li> Added more detailed error logging when JSON decoding fails happen in the hopes that it is more helpful in identifying & fixing issues.</li> Added fail-able Pull Request decoding so that successfully decoded pull requests are presented to the user and unsuccessful pull request decodings are logged.</li> Added a flash notice system to be able to inform the user in-app about things. Specifically, I used this to inform the user of pull request decode failures and provide a call to action enabling them to send a debug report so that we might address the issue.</li> </ul> Pra v1.3.0 2019-07-18T00:00:00+00:00 This release includes a mix of foundational features, small improvement features, as well as some bug fixes, performance improvements, etc. Ignore Organizations</h2> We added the ability to Ignore Organizations in addition to already being able to ignore a repository. This is an extemely useful feature that can help you gain focus by ignoring pull requests that belong to organizations you are a member of but aren't interested in reviewing. Quick Filters</h2> We also added a foundational feature, Quick Filters, in this release. It includes the Created, Assigned, Mentioned, and Review Requests quick filters while also laying the foundation for the future. Quick Filters enable quickly filtering the set of pull requests down based on a particular mindset. Below are a few mindsets and their associated quick filters. Trying to catch up on what last happened on one of your pull requests? Use the Created quick filter.</li> Trying to remember what pull requests you are responsible for? Use the Assigned quick filter.</li> Want to know who is talking about you? Use the Mentioned quick filter and see the pull requests that mention you.</li> Can't keep track of all the requests for reviews? No, problem see them with the Review Requests quick filter.</li> </ul> Bug Fixes, Performance Improvements, etc.</h2> In addition to the above changes we also did the following. fixed a small layout issue in Preferences</li> switched from GitHub v3 REST API to GitHub v4 GraphQL API</li> made some performance improvements</li> </ul> The most significant of these changes was switch to GitHub v4 GraphQL API as it facilitates getting the right data out of GitHub in a more efficient fashion. It also puts us on their latest API so that we don't have to be worried about the API being deprecated out from under us. Overall we think this release was an excellent step in the right direction and hope you enjoy the new features. As always we love hearing from you. So, don't hesitate ping us on Twitter</a> or email</a>. Pra - The App Store Audible 2019-02-22T00:00:00+00:00 It is Sunday, Feb 17th, 2019 at 10:58 pm and I have just submitted the Minimal Free Version of Pra</a> to the Apple App Store for review. I post in our team Slack channel that I've submitted and I cross my fingers as we wait for Apple to respond. Minimal Free Version</h2> It is probably worth explaining that our Minimal Free Version is pretty small in terms of functionality. It lets you do the following things: Manage an Account - Add or remove your GitHub account.</li> See a Feed of Pull Requests - See a continuously updating feed of pull requests you potentially want to review, aggregated across all repos you have access to, with the necessary data to quickly assess if you want to review one of them.</li> View a Pull Request - As you're looking through open requests, if you see one you want to review, you can quickly view the pull request details in your default browser.</li> Ignore Repositories - Select a pull request in the feed and ignore/hide the repository it's in from the feed of pull requests. Extremely helpful in reducing noise in your pull request feed.</li> </ul> We thought that the tool was already valuable in this state and we just wanted to get it in the hands of users to validate our assumptions. Would other people find this minimal state of the product valuable? If they don’t, it would be extremely useful to learn why not. If they do find it valuable, that is reassurance we are on the right path. Either way, we're interested in feedback that will help us decide which features to focus on and when, including what should be in the paid version and what should be in the free version. The App Store Response</h2> It is now Feb 18, 2019 at 7:22 PM and we just received a rejection response from Apple. Guideline 4.2.2 - Design The experience that your app provides is not sufficiently different from browsing a content aggregator web site. Next Steps While your app may facilitate access to content from a range of web sites, even when including native macOS features, the experience it provides is not significantly different from using Safari. Such apps do not include enough native macOS functionality to be appropriate for the App Store. </blockquote> Headspace</h2> To be honest, getting this response was frustrating and a little disheartening. Part of why I feel that way I think is because Apple's response is so generic and unclear. What do they consider native macOS functionality? The app is 100% written in Swift, doesn’t use any web views, and integrates with macOS Keychain and UIKit. There are also plenty of apps on the App Store that don't seem to include any more native macOS functionality than Pra</a>, even in its current, minimal state. Somehow those apps were allowed on the App Store. So a few of us (mostly me) start venting a bit about this in Slack, while we slowly start to calm down. Once I calmed down, we started throwing out ideas about what "enough native macOS functionality" could mean to Apple. We look at RSS Feed Reader apps because they are "content aggregators" and there are a ton of them in the App Store. Most of them provide a viewer interface for the posts, so we think maybe we need to add a pull request viewer in the app rather than just open the PR in the default browser. This is already in our idea backlog, so it's not out of left field. Adam suggested trying to find one or two features that would take minimal effort to build, but be unique to the desktop, to see if Apple would then accept the app as having "enough" native functionality. He threw out a few ideas: Some sort of notifications</li> Badge on dock icon</li> Banners/alerts</li> Menu bar icon</li> </ul> Another thought I had rolling around in my head was how much of the rejection was a side effect of our App Store copy not explaining the true value of Pra</a>? I was having a hard time homing in on exactly what we should do. So I went back to why we were trying to launch the app in the first place: to get crucial feedback and find out if other people find it valuable in its current state. At this stage, we're more interested in getting an idea of whether or not we are on the right track and gathering some feedback than we are interested in wide distribution. The Audible</h2> At this point, I started thinking maybe we should just launch via direct download from our website rather than via the App Store. This would allow us to get versions out quickly and gather feedback while we continue the review process with Apple. So, we hashed it out as a team in Slack and decided to put the download on our site (and start to gather feedback and iterate on the Minimal Free Version) while we continue to work with Apple to get clarification and get the app ready to resubmit to the App Store. The Non-App Store Launch</h2> Feb 21st, 2019 at 1:39 PM we launched a landing page with the direct download button to get the app, and shared it with small group of colleagues in another Slack team. Sam’s Feedback</h3> That afternoon I got feedback from a friend named Sam now using it and asking the following questions: How do I add more than one account? - We realized this was an indicator that our communication via Tooltip on the add account button was too subtle to properly communicate that the free version only supports one account. Also, it made us think we should probably have a mechanism at that point to allow people to submit their email address to join the mailing list to find out about releases and when multi-account support is added.</li> How do I connect to my Enterprise GitHub account? - This is a feature we talked about being part of the paid features previously as well. So, we took this as reassurance that there is at least some need for this feature.</li> </ol> Nico’s Feedback</h3> Later that evening I got feedback from another friend asking about what PRs are included in the feed. He wasn’t seeing a PR that he knew he could see in the Created at filter on GitHub. He thought it was because he is an External Collaborator on the repository, and it turns out he was right that that functionality was missing. So, I quickly made a ticket to resolve the bug and started working on a fix. I shipped the fix to Nico so he could test it and find out if it resolved his issue. Unfortunately, the new version still didn’t show the PR Nico was missing, but it did add the missing functionality of showing pull requests from repositories that you are an external collaborator of. Digging a little deeper with Nico, we eventually figured out that his missing PR was from a fork of a repository he no longer had access to. He thought it might not be worth the effort to fix (maybe this is just an edge case?). I responded that we probably wouldn't add this to the main feed view (because the main purpose of the feed is to show pull requests you might be interested in reviewing, not necessarily all the pull requests you can access). But, I also let him know that we were contemplating adding other feed views via quick filters, including a “Created” filter which would show this pull request. This would support a different user goal of checking in on the status of their own PRs, and if there is anything they can do to move them forward. Our conversation and interaction gave me reassurance that what we are thinking in terms of features, contexts/modes, etc. were right on track at least with Nico. The High</h3> Both of these interactions gave us a ton of value and amped me up. This completely made up for the feelings I had around the App Store rejection. The Second App Store Response</h2> Back to Apple — we decided to try and get clarification. So, Matt and Adam put together a response to Apple. Is there a way that we can get clarification on this rejection? Our app provides a significantly better experience than Safari, because it stores your GitHub credentials so that you can have a persistent and live-updating view of pull requests in your GitHub account on your desktop, without having to open a browser and log into your account. This view of pull requests isn’t available anywhere on the web today and having it persistently available via a desktop app provides unique value that you couldn’t create with just a web site. To do this in Safari would require going into each team in Github and going to each repository for each team to see what Pull Requests are open. This is a massive headache for a developer that is working in multiple teams with multiple repos per team. Happy to get on a call with you guys and walk you through how it works and what our roadmap is for enhancements to the desktop experience. </blockquote> Apple came back with the following response: 4.2 Design: Minimum Functionality (macOS) Hello, Thank you for your response. It would be appropriate to add native macOS functionality to be appropriate for the App Store. Regards, App Store Review </blockquote> What’s Next?</h2> Well, Apple's response didn't give us much to go on. So, we just need to put some thought into what “macOS Features” we can easily add (and that make sense), then submit to the App Store again. Meanwhile, we continue to tell more people about the Minimum Free Version, collect feedback, and iterate along. From this initial round of feedback, we've identified a few tasks: add pull requests for External Collaborator repositories to the feed</li> add support for joining a mailing list in the Account Preferences</li> add processing of pagination to GitHub API requests</li> </ul> From the App Store rejection we're planning to: review the App Store messaging and see if we can communicate the value better</li> figure out what native macOS features we think make sense to add in order to get it approved for the App Store</li> </ul> Overall, I feel very confident that the decision to release the app outside of the App Store this early in its lifespan was one of the best things we could have done. The feedback is invaluable and each iteration we make improves the app. In retrospect this really cost about a day's worth of work to research (and implement) the proper way to package and release the app outside of the store. In the scheme of things, a relatively small price to pay for the feedback we obtained in the first round. Create macOS SideBar in Swift 2019-01-10T00:00:00+00:00 Last night I started digging into building a SideBar for Pra</a>. Sadly, I struggled to find a decent example of how to programmatically create a SideBar in Swift for macOS. After a little bit of trial and error the following is the working solution I came up with. import Cocoa extension NSSplitViewController { convenience init(sideBarViewController: NSViewController, contentViewController: NSViewController) { self.init(nibName: nil, bundle: nil) let sideBarSplitViewItem = NSSplitViewItem(sidebarWithViewController: sideBarViewController) let contentSplitViewItem = NSSplitViewItem(viewController: contentViewController) self.addSplitViewItem(sideBarSplitViewItem) self.addSplitViewItem(contentSplitViewItem) } }</code></pre> Pra: Status Update 001 2019-01-07T00:00:00+00:00 This is the first status update of the Pra macOS application. I figured what better time to do this than right after 2018 ends and 2019 begins. My goal with this is to give you an idea of where I am with the development, give you some insight into the things I am doing and some of the decisions I am making along the way. If you would like a more real-time feed of what is happening and want to provide feedback you can always follow me on Twitter @drewdeponte</a>. Initial Setup</h2> The first step I took with this application was to start a basic macOS application. Immediately following that I: replaced all the Interface Builder xib stuff with straight up source code because I prefer to develop apps without Interface Builder or xibs</li> built out the Main Menu, Main Window, Preferences Window, and the Preferences Window’s tab interface</li> built out the Account Management Interface</li> built out Main Window for Pull Request list</li> </ul> Building out the basic skeleton of App in terms of flow is part of the outside-in approach I like to take when developing applications. Basic App Skeleton</figcaption> </figure> Making Account Management Real</h2> Once I had the skeleton of the macOS App together in terms of windows, etc. It was time to make adding GitHub accounts and the persistence of them and their associated API keys a thing. I also built out a very basic stubbed out pull request list view in the main window as I knew that the next step would be to actually add fetching and displaying of pull requests. Note: I decided to implement GitHub integration initially because it is what I primarily use and I believe it has the largest user base. Now with some content</figcaption> </figure> Account selection</figcaption> </figure> Real GitHub integration</figcaption> </figure> </div> Making Pull Requests Real</h2> After adding basic Account Management I proceeded to drive toward making it so the App could fetch, present, and support basic interaction with Pull Requests. My goal with this phase of development was to get the minimal aggregation functional, interaction and UI implemented so that we could try it out and see if we felt it was functional at all in this rough state. So, I implemented the following: Fetching & aggregating Pull Requests from accounts</li> Scheduling of Pull Requests refresh at 5 mins</li> Open Pull Request by double clicking</li> Refresh Pull Requests on account addition/removal so user doesn’t have to wait ~5 mins after adding an account to see their PRs</li> </ul> The most basic list of pull requests</figcaption> </figure> Making Things Functional - Round 1</h2> At this point I distributed it to Ryan</a>, Matt</a>, Adam</a>, and Claude</a> so they could start running it to see how functional/useful it was at this point. We decided that it was functional, supporting our theory that we believe there is value in this App. However, there were some immediate things that we thought would make it easily more valuable. So, I added: Displaying Repository name into the pull request list</li> Ability to ignore Pull Requests from repositories by right clicking on a pull request and selecting ignore repository.</li> Adding organization, description & avatar to Pull Request list - Note: I did this without caching of avatar images, etc. so that I could get feedback quicker, with and with less investment.</li> </ul> It's starting to look like a real app now</figcaption> </figure> Some Polish - Round 1</h2> I sent the latest build to Ryan</a>, Matt</a>, Adam</a>, and Claude</a> and we started using it again. After a day or so of use the feedback was pretty unanimous. The added features aided significantly and we felt we were on the right path. However, we did identify a number of small things that just didn’t make it feel polished enough or didn’t make it feel like a real product. I like to tackle these types of issues as I develop an application for two main reasons. it is a pain to have to worry about them all at the end</li> it is hard sometimes to not have feedback swayed by some of these polish things</li> </ul> This lead me to start designing an icon for the App. It is weird how much of mental game not having an icon for your App is. But, once you have one (even if it isn’t the greatest final version) it does a huge amount to make you feel like you are moving forward and that your app is becoming a real thing. So, I designed the first draft of the icon. I took ideas from both the Pull Request Icon that GitHub uses and the Git logo, threw them in a blender and came up with the following to get started. App icon, v1</figcaption> </figure> I also added Microsoft App Center integration for Analytics Tracking and Crash reporting. These are crucial as a feedback mechanism as I distribute the app internally and as I prepare to distribute the free version publicly. Beyond that, I made cut, copy, paste, undo & redo work. This was especially important in the OAuth WebView because when you would try to fill out the OAuth form you were not able to paste your password from a password manager. Extremely annoying to say the least. So I had to fix that one. Adam Korman</a> had some time to catch up with me and work off of the various draft variations of the app icon I did, this time taking Apple macOS App Icon guidelines into account (which I completely ignored, whoops) and we ended up with the second draft of the Pra macOS App Icon. App icon, v2</figcaption> </figure> Making Things Functional Round 2</h2> After adding some polish and sharing the build with Ryan</a>, Matt</a>, Adam</a>, and Claude</a> and using the app for a bit we quickly identified a few other changes that we thought would help improve the value of the core functionality. So I added assignees to the Pull Request view when they exist for a given Pull Request. This gives people the ability to quickly determine from the list of Pull Requests which PRs have assignees and who they are. This aids with the user deciding if it is a PR they need to review and provide input on or not. More polish</figcaption> </figure> I also added Pull Request body Markdown rendering. I did this for two reasons. the bodies of GitHub pull requests are often written in Markdown as GitHub supports Markdown rendering.</li> the Pull Request bodies were coming back with hard breaks, and this made word wrapping look weird.</li> </ul> Markdown rendering allows us to interpret paragraphs, ignoring hard breaks, making word wrap work as expected as well as support Markdown formatting used in Pull Request bodies. Now with Markdown rendering</figcaption> </figure> Conclusion</h2> We have been using the app in this state for a little bit now and have some more polish changes I need to make. For instance: use slightly smaller fonts</li> support dark mode/light mode explicitly so that we have better control of color and theming</li> </ul> I also have a number of features I need to add before I feel the initial free version will be ready: figure out a good in-app feedback mechanism and make sure people can trivially find it and know we want feedback (feedback is key to success)</li> add welcome tutorial that walks user through initial setup and educates them on how it is intended to be used</li> add buttons to the UI to support interactions with pull requests so that the interactions are more discoverable than mysterious right clicks, etc.</li> add relative time to the pull requests so the user can see how old PRs are</li> add avatar caching to make the app feel more responsive</li> restrict the account management to 1 account and provide in-app education about why user can only have 1 account in the free version</li> </ul> Outside of that I have to do the following: build out a landing page</li> get our Privacy Policy together</li> get our Support page together</li> write up all the App Store marketing material</li> </ul> This will get us to an initial App Store launch at which point we can start promoting the app and talking about the app to get more users and get more feedback while we start working on the premium tier of the application. At least that is the high level plan as of right now. Pra: Thinking about Money 2018-12-19T00:00:00+00:00 While I am waiting to collect information from the survey</a>, I started thinking about how I could make money with Pra. I thought about it before at a high level. But now I have started thinking about it in more depth. Expectations</h2> It is important to understand that my expectations aren't that this product be some sort of unicorn. I share the mindset of Paul Jarvis</a> — I want to build a successful, sustainable business rather than shoot for explosive growth — and we largely operate our current boutique consulting/contracting business that way. Paul just launched a book titled Company of One</a> where he talks about this mindset. If I boil it down, my expectations for Pra are to be something that: actually helps people</li> I can be proud of</li> isn't overly complex or requires massive amount of effort to develop</li> I can sell and make a relatively small amount of money from</li> I can use to provide content to start building an audience</li> </ul> How Do I Make Money?</h2> After talking in depth about this with my co-workers, Ryan and Adam, it seems like I want to have a free tier that provides "real" value. Ideally I want people to find enough value in the free version that they use it over GitHub (or any of the other providers) directly and are willing to tell other people about it. Then, to make money, I need to find compelling additional features that at least some people would be willing to pay for. The natural organizational entities are accounts, organizations, and repositories. Stances</h3> There are two main stances I have thought of related to this product that I could take: Core Value Paywalled. - In this stance, I would provide a free version of the app that wouldn’t do any aggregation. For this to work, I think it would have to be an app that works similar to GitHub, where you get access to the repos belonging to a single account and the ability to click on one and see the pull request for that specific repository. I believe it would also need to have some additional (free) features that make it more useful than just using Github itself. Then, I would have a paywall gating access to the aggregation feature. I would then have to provide some sort of trial experience to let the user experience the value of aggregation. The concern with this stance is that it would reduce the number of people willing to try the product out because they are hit with a paywall/trial right off the bat.</li> Core Value Free - In the core value free stance, I would provide the aggregation for free within a single account. Then the stuff behind the paywall would be N accounts, as well as the set of tertiary features that make the overall experience better. The high-level thinking about this strategy is that if it is valuable enough, I may be able to get a larger number of people using the free version (since they get the core value for free). I would then have a channel through paywalled tertiary features that I would communicate on a personal level about who I am, what our mission is, what they get, and talk about it in terms of supporting our mission. The thinking is I could get a much wider net of free users to build our following for communicating other maker tools, etc. in the future, and some subset of people will pay for a Pro version. The concern with this approach is that if I provide the core value for free, there's less additional value (that people would pay for) in the Pro version.</li> </ul> My current stance is that I am not 100% sure which route to take. My instinct is that I should take the Core Value Free stance as it facilitates growing a potentially larger following for other products in the future, and I can envision how to make it happen. Also, in the Core Value Paywalled stance I would potentially have to build out a larger more complicated version of the app prior to the paywall being a thing, as it wouldn’t make much sense for people to use it if it was a worse tool than what GitHub already provides. The above leads us to the following questions: Do I charge by the repository and limit the free version to 1 repo, or maybe some fixed number of repositories like 3?</h4> After hashing it out I think the answer to this question is no. I don’t think it provides/communicates/shows the value of aggregation when it is limited to simply 3 repositories. Do I charge by the organization?</h4> Well, I could. However, the largest use case of people are probably people that have one primary Organization (the org of the company they work for), and their personal organization. The question then becomes is there enough value of aggregation seen for them to then purchase a subscription just to get their personal org added in, or would they just deal with the annoyance? After talking about it with Adam and Ryan I think most people would just deal with the annoyance. So, the answer is no. I also need to keep in mind that I want the largest possible group of people using and finding the free version valuable so that I can convince them to pay for even more great value. Do I charge for multiple accounts?</h4> Initially the thinking around this was that I charge for multiple accounts, as it is probably a niche subset of the market (consultants/contractors). However, I know personal developers who have their personal repos in BitBucket and their company repos in GitHub. So, I am not sure where I net on this at the moment. It still needs some consideration. So Where am I?</h3> If I don’t charge based on core organizing principles what do I charge for? Right now the thinking is that I would charge for features. For instance, provide some of the Quick Filters for free and some of them would require you to be a subscriber. I could also make the search functionality, keyboard shortcuts, enriched PRs, etc. paid features. Principles of Purchase</h3> One thing I don’t like is the complexity around purchases and nickel and diming people. So I think it should basically be two tiers: Free – Roughly feature parity with seeing PRs in GitHub, with the added value of the aggregated views and some other nice to have features</li> Pro (probably a monthly subscription) - All The Features. I think that getting people to pay is as much about access to the features as it is finding supporters around our mission of helping makers, and who we are. So they aren’t just buying a thing, they are supporting a small boutique group of real people they can identify with.</li> </ul> One thing I have talked about as a possible 3rd tier would be Enterprise, which would potentially get you integrations with GitHub Enterprise, etc. The thinking around this is some individuals might pay for it because they have become dependent on Pra and love it. But I think the real money with that is in group licensing to companies that use Enterprise GitHub, etc. Conclusion</h2> Although I don't have all the answers, my instinct is still to take the Core Value Free stance. I do believe thinking through some of this prior to getting the results from the survey back is a good thing. It helps me look at the data I get back in ways I may not have looked at it originally. Hopefully, the results of the survey will help clarify answers to some of these questions. Pull Request Notifications: an Anti-Pattern 2018-12-11T00:00:00+00:00 Back in 2013, I was lead of a software development team at ReachLocal, Inc. At the time we were using GitHub for open source projects and Atlassian Stash for our private Git repositories. We required at least one reviewer to review each pull request before it could be merged into the mainline. We also generally didn’t assign specific people to pull requests as reviewers because we wanted to promote knowledge sharing throughout the company. So, people had to go grab a pull request or two every once in a while and review them. Problems</h2> Email notifications didn't really work well in this scenario and they have a tendency to get lost in peoples’ inboxes. So naturally we progressed to annoying our teammates via our company chat system (Slack, now) until someone eventually reviewed the pull requests. It didn’t take me long to realize that we had fallen into a communication anti-pattern — people didn’t need instant feedback on their pull requests. Also, the chat messages announcing pull requests ready for review generally got lost in the mix of the other chat messages, and the author would end up posting a message again and again to get peoples’ attention and hopefully get a review. This was also having the negative impact of breaking the developers on the team out of “deep work” and costing a huge amount in terms of context switching. At the same time, we were having problems losing track of pull requests across a number of repositories (and across different Git repository management systems). This would happen all the time, as we had different repositories for various libraries, services, and micro-services we had built out for the systems my team was responsible for. Since the only way to proactively check for pull requests to review was to go to each individual repository and check the pull request tab, it's not surprising that they would fall through the cracks. Baby Step</h2> The culmination of these problems drove me to try and come up with a solution. Something that would ideally show me all of the pull requests that I cared about seeing, none of the ones I didn’t, and exactly when I wanted to see them. As a developer, my natural tendency is to build a software application to solve my problems. So that is exactly what I did. I created an open source terminal based curses application in Ruby called pra</a> (Pull Request Aggregator). This app was a great step in the right direction and many people continued to use it as we went on to work at other companies. But, it was definitely lacking in certain areas largely due to the limitations of curses. For instance, if it had GUI then it could provide more information in a more concise view as well as provide other functionality around viewing specific types of pull requests, etc. State of the World</h2> Jump forward about 5 years to December 2018 and sadly not much has changed in the pull request world. On my front, I am the co-founder & co-owner of a very small (5 of us in total</a>) boutique Software Product, Design, and Development Consultancy, Uptech Studio</a> where our mission is to provide services and tools to Makers that help them bring their dreams to life. The team here at Uptech Studio</a> was going through a list of potential products we could build. Looking at pull requests and the space around them, we still believe there is a gap in the workflows and the lack of tooling around them. Therefore we have decided to tackle this problem space. We believe building a GUI version of pra</a> is a great place to start and iterate from. Although we have envisioned a much more feature rich application that we think will solve other problems, we are approaching a point where we plan to release a free version that even in an early form would be useful and valuable. To get a taste, below is a snapshot of a very early, very rough internal build. The Future</h2> Being a very small boutique (yet experienced) product, design, and development team, we know better than to just build something and hope that our vision is right. We need to vet our theories before we invest lots of time and money into making them a reality. This is where you come in. I know it is a lot to ask, but we need help vetting some directions we are thinking about taking the product. So, we have put together a small single page survey</a> to help guide us on some of these decisions. Fill Out the Survey</a></div> If you have read this and have experienced some of these issues and would value a product focused on solving problems in this space please provide your email below showing your support for this product. We promise we won’t spam you. We will provide you with details about launches, ask for feedback about certain features, and invite some of you to alpha or beta releases. Sizing and Optimizing Images for iOS 2018-11-09T00:00:00+00:00 Why?</h2> Sizing and Optimizing images for iOS can have a huge impact on performance of your app and the overall user experience. Therefore, it is important to take the time to figure out the proper sizing for your images and to make sure that they get appropriately resized and optimized for your needs. Understanding your constraints</h2> I like to start by figuring out the devices that I am interested in supporting. I was recently working on an app where we wanted to support the iPhone 6 all the way up to the latest iPhone Xs Max. So to figure out the sizes of the images I needed some context as to what size in points the screens on those devices are and at what resolution they get rendered. I use The Ultimate Guide To iPhone Resolutions</a> all the time for this type of reference. In this case since I am supporting iPhone 6 as the lowest size which is 375 x 667 points and the iPhone Xs Max as the largest size, 414 x 896 points. If you look at all the point sizes between that range of phones. There are really two different aspect ratios I need to support 0.46 for the iPhone Xs Max, iPhone Xr, iPhone X, iPhone Xs, and 0.56 for the iPhone 6+, 6s+, 7+, 8+, 6, 6s, 7, and 8. This means that I am most likely going to have to do some cropping depending on which device the images are loaded on. Image space in points</h3> Now that I have some context in terms of max size and min size. I need to figure out what the point size of the actual image space is going to be. I approached this in this scenario by adding a log line to the app that logged point size of the frame of the image area. For this particular app</a> I am implementing a carousel of profile pics of people participating in the application. At the moment I had an iPhone X and a iPhone 7+ with me. So, I built the app and ran it on both the iPhone 7+ and the iPhone X to get the point size of the profile cards on those devices from the logs. Device </th> Size in Points </th> Aspect Ratio </th> </tr> iPhone 7+ </td> 254 x 416 </td> 0.61 </td> </tr> iPhone X </td> 215 x 434 </td> 0.49 </td> </tr> </table> From there I could deduce what the point size of the Profile Card would be on the iPhone 6 and the iPhone Xs Max by looking at the difference in points between the screen sizes. For example the iPhone 6 screen size is 375 x 667 and the test device I had was an iPhone 7+ at 414 x 736 which is 39 points wider and 69 points taller. Knowing that, I can apply those difference to the iPhone 7+'s Profile card point size of 254 x 416 resulting in the Profile Card being 215 x 347 points on iPhone 6. The same concept can be applied going from The iPhone X Profile Card point size to the iPhone Xs Max Profile Card point size. In this case the iPhone X Profile Card point size is 215 x 434 and the there is a difference of 39 x 84 between iPhone X and iPhone Xs Max point sizes. When applied this results in the Profile Card being 254 x 518 points on iPhone Xs Max. Device </th> Size in Points </th> Aspect Ratio </th> </tr> iPhone 6 </td> 215 x 347 </td> 0.619 </td> </tr> iPhone Xs Max </td> 254 x 518 </td> 0.490 </td> </tr> </table> All of that was to understand the point size dimensions of the smallest image and largest image spaces that I want to support and make sure they look beautiful. From this data I see one very important thing. The aspect ratios are not the same. This means that there is going to be some cropping happening. Where to crop</h3> In this case I was prepping these images for profile pictures of people and they were sending their headshots to us. Most of these headshots had one thing in common, the persons face was toward the top of the image and relatively centered horizontally. For this particular type of content it is safer to crop off the sides of the images generally, rather than cropping off the top as I don't want an image to crop someones face. If I were for example to make the size of the images match the largest devices height, on the smaller devices with the different aspect ratios peoples heads would get chopped off like you see in the example. Because I don't want to lose any of the tops of the images. I needed to use the smallest height and the widest width from our device extremes for the image. This covers me in all the sizes between and will make it so only the sides get cropped. Yes, it does mean that I have some upscaling happening on the larger devices. But, in this case we did this optimization and tested it and found the upscaling looks fine as it is relatively minimal. In my case that meant that the image size in terms of points that I wanted to produce for these profile cards was: Universal Image Size (points) </th> </tr> (Longest Width) x (Shortest Height) </td> </tr> (iPhone Xs Max Width) x (iPhone 6 Height) </td> </tr> 254 x 347 </td> </tr> </table> Getting the Resolution Right</h3> Now that I have an understanding of the image space I need to worry about in points for the image. I need to figure out what size the actual image should be in pixels. Given that points are the representation in iOS land when you are developing and pixels are what points get translated into and rendered out to on the screen. We need to have an understanding of how these devices render pixels out. Luckily The Ultimate Guide To iPhone Resolutions</a> provides this information for us as well. If we look at the iPhone Xs Max we can see that it renders to the screen at 3x. This means at a value of it's screen's point size multiplied by 3. The iPhone 6 on the other hand looks to only render at 2x. Meaning 2 times it's point size. So, I am now presented with a choice. Option one, pick the 2x rendering of the iPhone 6. This would result in a pixel size of 508 x 694. This size would leave a pretty big gap for the larger devices to fill with upscaling. The other option is to use 3x rendering of the iPhone Xs Max which would result in a pixel size of 762 x 1041. This seems like a much more reasonable size for the larger devices and it will just be scaled down to fit on the smaller screen devices like the iPhone 6. Therefore, I opted for the later option of a pixel size of 762 x 1041. I took this size and added an extra 30 pixels to both the width and the height as I am planning on using some image techniques later on in the app that would require having that overage. So I really resulted in wanting images at 792 x 1071. Batch Scaling Images</h3> Now, that the hard part of figuring out what the best size for the image is. The easy part can begin. I found this wonderful little app named Resize Sense</a>. It provides the ability to batch apply various transformations to images. In this particular case it will help me fit, scale, and crop the images that people sent me to the appropriate size. It is worth noting that it isn't the most intuitive app but it does work well. So, experiment with it a bit and it will be well worth it. The first thing I did after launching the Resize Sense</a> was to check out the Preferences. I quickly noticed that it has a Default image resolution: option which is defaulted to 300 pixels per inch. This got me wondering what the pixels per inch on the iPhones where. A little googling lead me to 458 pixels per inch so I updated that preference. I am not sure why this is a global setting. It seems like it should be a setting in the presets but I haven't seen it in there so I just went with it as a global setting. After that I discovered that I need to build what they call a Custom Preset. I did this by clicking the Manage Presets button and defining a preset as follows. Note: According to their documentation it is important that presets have unique names. You will notice that the pixel dimensions of the image preset are the same ones we determined above. After creating the preset I needed to select it and click the Apply selected preset button which then moved it to the applied presents list. And then I went and selected the first batch of headshot images people sent me and dragged them into the lower right pane of the app and selected one of the images. This applied the transformations and showed me a preview of the resulting image in the upper right pane. From there I simply selected Save All images from the File menu and selected a destination for the files, loaded the saving preset from the custom preset I made earlier and clicked Save images. This batch processes all the images I gave it and produced a resulting set of images that had been sized appropriately, cropped, etc. The only issue was that the JPEG image optimization in Resize Sense</a> isn't very good. This isn't actually a huge problem as there is a community defacto standard that is used which is easy and we know is good. Therefore, I ran the batch scaling, etc. to output at 100% JPG quality as I knew that it would be optimized later. Optimization</h3> Now, that I had a batch of appropriately sized images for the App's Profile Pictures I wanted to compress and optimize them to try and reduce their file size even further. It is worth noting that when I started this process I was dealing with multi-megabyte file sizes for these images. So, next I opened ImageOptim</a>, a wonderful little macOS application that is as simple as they come, and I configured the preferences to be as follows by clicking the little gear icon. Then, I dragged all the image files that were output by Resize Sense</a> and ImageOptim</a> did it's thing and I ended up with great looking images that were now sub 100kb. This is a huge game changer for the performance of the application as these images are downloaded over often slow cellular networks. So, please make sure you take the time to properly figure out the right image sizes for your app and please optimize them. Inline Links in Text in iOS Apps 2018-10-19T00:00:00+00:00 Today, I was working on a client project. As usual a couple of designers had done some high fidelity designs in Sketch</a> that I was riffing off. In the designs on the Sign Up screen near the bottom there was the classic notice stating that by signing up you are agreeing to the User Agreement and Privacy Policy. There are two interesting things about this. First and foremost the phrases User Agreement and Privacy Policy are visually called out using blue like they would be if they were anchor tags (a.k.a. links) in a website. Secondly, both these phrases are inline with other text. If I was doing this on a website this would be trivial as I would just put two anchor tags inline in a paragraph tag with the rest of the text. However, on iOS things aren’t so straight forward. Assumptions</h2> My assumptions going into this were that I was going to have to somehow detect a tap on each of those phrases and then present a new modal View Controller containing a web view that loads the User Agreement or Privacy Policy respectively. I knew off hand how to do all of these things except for the detection of the phrases being tapped. Though, I did have enough knowledge to know that with some work it could be done. Research</h2> I started out searching for “UILabel tappable text”. It led me to a variety of half baked attempts at solving this with code that seemed way more complicated then it should need to be in my mind. I am talking like 60+ line solutions. So, I kept digging and stumbled across a post referencing the existence of some sort of NSAttributedString</code> link feature. I thought this was interesting but sadly the post did not give me much direction in terms of actually making it happens. So, I searched a bit more and found an example of applying the NSAttributedString.Key.link</code> attribute to an NSMutableAttributedString</code>. An example is as follows: let attributedString = NSMutableAttributedString(string: “by signing up, you are agreeing to Float’s User Agreement and Privacy Policy.”) attributedString.addAttribute(NSAttributeString.Key.link, value: “https://example.com/policy”, range: NSRange(location: 43, length: 14)</code></pre> When I applied the above and then assigned it to the UILabel</code> ‘s attributedText</code> property I got excited because it resulted in the User Agreement phrase showing up as blue with an underline. This was close to what I wanted design wise and gave me hope that it might actually do a thing. Sadly, tapping on what I believed was a link had no impact. So, I dug further and tried a number of various properties on UILabel</code> such as isUserInteractionEnabled</code> but no luck. After searching a bit more I stumbled upon an article that referenced further some delegate methods from UITextViewDelegate</code> that people were using with the NSAttributedString.Key.link</code> to handle opening them in a browser. This gave me a new path to explore. UITextView</h2> So, step one I converted my UILabel</code> to a UITextView</code>. When I initially did this the view was UITextView</code> was not visible and inspecting the view I could see that there was layout constraint issues. So, I gave it a hard coded height using auto layout constraints. Magically, it was visible and the layout constraint issue went away. So, now I knew that it was not being sized based on it’s content. After a quick search I was able to find out it is because it has a isScrollEnabled</code> property that is by default set to true</code>. So, I flipped it to false</code> and got rid of the explicit height constraint. Like magic it was there and visible just liked I wanted with the correct styling. Next I attempted to tap the link. But, to my surprise the keyboard popped up. Of course that is because there is another property isEditable</code> that is by default true</code>. So, after flipping that to false</code> when I tapped the link it actually opened Safari to the URL of the link. Finally some success! At this point my UITextView</code> was configured as follows: textView.isEditable = false textView.isScrollEnabled = false textView.backgroundColor = UIColor.clear</code></pre>Breaking Assumptions</h2> At this point it was actually functioning though in a different fashion than I had originally assumed it would. This made me question my initial assumptions of the need for having a separate view be presented modally containing a web view that would load the content. The default behavior I was seeing is actually pretty much inline with those concepts. It is basically modally presenting Safari with the content loaded in Safari. My immediate next thought was well how easy is it for the user to get back to the app where they left off in either scenario. In the separate view scenario they would have to hit a close button to dismiss the modal view and go back to the Sign Up view. In the Safari scenario the user simply hits the back to last app button in the upper left. So, turns out that the default behavior experience feels great, is minimal effort and models how most apps on macOS handle links as well. At this point I am pretty confident this the ideal path and I am driving full force ahead. Another Problem</h2> Turns out there is one problem. I have been localizing this application up to this point using NSLocalizedString</code>. Looking at the example: let attributedString = NSMutableAttributedString(string: “by signing up, you are agreeing to Float’s User Agreement and Privacy Policy.”) attributedString.addAttribute(NSAttributeString.Key.link, value: “https://example.com/policy”, range: NSRange(location: 43, length: 14)</code></pre> We are clearly coupling the link to a specific starting location and length within the string. So, sadly this breaks localization because different languages will have different starting locations and different lengths, and with some googling there is no built in way for localization to support NSAttributedString</code>. Solution</h2> But, wait there is still hope. At least in my opinion the above linking strategy is by far the best solution I have been able to find. So, if your app does not need localization you can use it straight away. If on the other hand your app does need localization. Don’t worry I have an idea. The thought is to actually use a subset of Markdown</a> in the localized string content and then write a method that will fetch a NSLocalizedString</code> and parse the subset of Markdown</a> into an NSAttributedString</code>. This would in theory resolve the localization issue and potentially give localization a decent bit more functionality in terms of representing text. In fact this can trivially be done with existing Markdown parsing libraries such as iwasrobbed/Down</a>. You could use it as follows to get your attributed string with links. let down = Down(markdownString: NSLocalizedString(“USER_AGREEMENT_NOTICE”, comment: “User Agreement Notice”)) let attributedString = try? down.toAttributedString()</code></pre> The only down side to this approach, at least with this Markdown library, is that the NSAttributedString</code> it produces has the links styled with the NSAttributedString.Key.link</code> attribute as well as the underline style. This makes it look different than the designs. With a quick read of the iwasrobbed/Down</a> README.md and some spelunking of the DownAttributedStringRenderable.swift</a> file I quickly discovered that the toAttributedString()</code> method has a toAttriubtedString(stylesheet:)</code> variant that allows passing a String</code> of CSS in to control the styling. In my case this just meant that I need to remove the text-decoration</code> on anchor tags. I thought about making a library to support this. However, now that I have seen how trivial it is to implement I decided I would just implement a helper method to do this. The helper I now write looks as follows: func localizedMarkdownString(_ key: String, comment: String) -> NSAttributedString? { let down = Down(markdownString: NSLocalizedString(key, comment: comment)) return try? Down.toAttributedString(stylesheet: "a { text-decoration: none; }") }</code></pre> That’s it! Relish in the beauty of localized Markdown and NSAttributedString</code> coming together so harmoniously to solve this problem. Decided what to Build 2018-03-11T00:00:00+00:00 We were struggling to sell one of our side projects OrangeCal</a> while also selling our consulting services as they were for completely different audiences. So, we decided to build and sell products that fit with our clients (software product makers) in the hope that we would find some overlap in sales and at least streamline mental focus for us in terms of sales. So we looked at the various product spikes we had created as side projects / open source projects and laid them all out on the whiteboard and had a team discussion to weigh pros & cons of each one. We ended up netting on building a product based on an open source project I built as a ruby command line tool called Pra CLI</a> (Pull Request Aggregator). At the time it had approx 15,419 downloads. We also knew that some of the developers on our teams at the last few companies had used it and others didn't specifically because it was a command line tool. So we decided that was enough interest for us to build a macOS Desktop MVP and see if we could get some traction with it. The plan was to essentially provide the same core functionality in a nice macOS Desktop app and solve some of the problems that prevented people from using the command line version. We figured we could expand on it with other features and figure out the paid product beyond that. I had written down a small list of possible features that the guys seemed to think were good. So, I started coding. Git Commit and Tag Signing 2017-08-04T00:00:00+00:00 I believe that there is a large gap in current development practices in terms of association of identity and source code. This is something that a large percentage of people have been ignoring for quite some time. I was one of those people until recently. I started asking myself the question: How hard would it be for someone to write code and claim me as the author, basically forging code in my name? I realized that it was as simple as someone setting their authorship email and name in their .gitconfig</code> to the values of my email address and name. The more I thought about this, the more it concerned me, and the more it seemed there must be a solution to this problem. This especially becomes a problem when you start talking about regulations around software development and the legal responsibility involved with software that is tightly coupled to human life and well-being, for example software in self-driving cars. The Solution</h2> Coming from a security background, I knew that a digital signature could solve this problem. So, being familar with GPG</a>, an encryption and digital signing tool that is often used to digitally sign and encrypt emails, I decided to search and see if Git</a> supported integration with GPG</a>. Sure enough it does, specifically for signing commits and tags. Knowing this, the next step was to setup and configure my machine, which is running macOS Ventura Version 13.0.1. How I Did It</h2> The following is a breakdown of the final set of steps I took to get Git</a> and GPG</a> functioning together. It took me a number of tries to get to this point, as numerous articles didn't seem to work for me with version 2.3.8 of GPG</a>, which is what I am using. It is also probably worth mentionining that I am using Git</a> version 2.38.1. Install Dependencies</h3> The first thing is to make sure that you have the necessary dependencies installed. I did this via Homebrew</a>, and recommend the same. brew install gnupg pinentry-mac</code></pre> The above installs gnupg</code> (a.k.a. gpg</code>) the tool set for digital signing, and pinentry-mac</code>, a tool to securely prompt and manage password/pin entry. GnuPG Key Pair Generation</h3> Once you have the dependencies installed, assuming you don't have a GPG</a> key pair already, you need to create one. This is done by running the following command and following the on screen instructions. gpg --gen-key</code></pre> Next, I recommend backing up your key pair. This is important, as you probably won't want to create a new identity if you get a new computer. I recommend backing it up in a physically secured space, like a safe. Given the depth and size of debate around proper key management, I will simply point you to the NIST Recommendation for Key Management</a>. The crucial thing is that you never ever share your private key, otherwise all this effort is pointless, as they would be able to forge your commits and tags. Get your GPG Key Identifier</h3> Once you have your key, the next step is getting your key identifer. This is needed to configure Git</a> properly later. The key id is actually a component of the fingerprent of the public key certificate, the thing holding all your subkeys and meta data that are used for signing and encryption. Specifically, the "Key ID" (a.k.a. Long Key ID) is the last 16 characters of the fingerprint. You can get it by running the following: gpg --fingerprint <email you tied to the key during generation></code></pre> The output should look something like the following: pub rsa2048 2017-07-28 [SC] [expires: 2019-07-28] 678A A67F A701 CFA7 D666 BCE8 41C5 D2C6 E5AF 944C uid [ultimate] Andrew De Ponte <cyphactor@gmail.com> sub rsa2048 2017-07-28 [E] [expires: 2019-07-28]</code></pre> In the above output the following is the fingerprint: 678A A67F A701 CFA7 D666 BCE8 41C5 D2C6 E5AF 944C</code></pre> The last 16 characters are: 41C5 D2C6 E5AF 944C</code></pre> If you remove the spaces you have the Key ID, which you will need very shortly: 41C5D2C6E5AF944C</code></pre> Note: It is ok to expose your fingerprint. It is part of the public key certificate and is something that you would share with someone to prove that the public key they have for you is actually your public key. Tell Git about your GPG Key</h3> Next you need to tell Git</a> that you have a GPG</a> key and to use it to sign your commits and tags. This is done by running the following: git config --global user.signingkey <your gpg key id> git config --global commit.gpgsign true git config --global tag.gpgsign true</code></pre>Configure GPG</h3> Then you need to configure GPG</a> appropriately to work on macOS. In order to do this you need to create the ~/.gnupg/gpg-agent.conf</code> file, with the following content: pinentry-program /usr/local/bin/pinentry-mac</code></pre> The the above config tells GPG</a> to work with the pinentry-mac</code> program we installed at the beginning, allowing it to properly prompt us for the password when needed. Testing It Out</h3> Now that everything should be set up, you need to test it out and verify that everything is working properly. Below are the steps I took to create a test repository with an initial signed commit. $ mkdir -p /tmp/test-git-signing $ cd /tmp/test-git-signing $ git init . $ echo "Hello" > README.md $ git add README.md $ git commit -m "Initial signed commit"</code></pre> You will probably be prompted by pinentry-mac</code> to enter your GPG</a> key password. Note: You can check the box to have it store and manage your password in Apple Keychain. This will make it so you don't have to enter your password all the time. Now you can check out your signature in all it's glory by doing the following: $ git log --show-signature</code></pre> The output should look something like this: commit 1a5fed0757e2c8600318595f1681cf508033e72f (HEAD -> master) gpg: Signature made Sat Aug 5 01:30:43 2017 PDT gpg: using RSA key 678AA67FA701CFA7D666BCE841C5D2C6E5AF944C gpg: Good signature from "Andrew De Ponte <cyphactor@gmail.com>" [ultimate] Author: Andrew De Ponte <cyphactor@gmail.com> Date: Sat Aug 5 01:30:43 2017 -0700 Initial signed commit</code></pre> The above shows you that the commit is signed and that the signature is verified. If it was a commit by someone else and wasn't verified, maybe because you didn't have their verified public key in your GPG</a> key ring, it would look something like the following: gpg: Signature made Wed Sep 13 02:08:25 2006 PDT using DSA key ID F3119B9A gpg: Can't check signature: public key not found error: could not verify the tag 'v1.4.2.1'</code></pre>Great. So how does this help?</h3> This makes it possible for people to verify via cryptography that you are actually the person that authored the commit or tag. If they are signing their commits and tags it also allows you to do the same. In fact, Git</a> even allows you to have it reject merges and pulls if the commits are not signed by a verified person. To see more details on this, check out the Git Tools Signing Your Work in the Git SCM Book</a>. To really understand exactly how that is done you have to understand that you have two keys, a public key and a private key, and that they are related. You share, and generally publish, your public key. This is because the public key is used by others to do two things. First, to verify signatures you have made with your private key, and second, to encrypt information to make sure you are the only one that can access/decrypt it using your private key. Publish Your Public Key</h4> You need to make sure that you have given people your public key, that you have the other collaborators' public keys, and that everyone has verified everyone's public keys. To facilitate sharing public keys there are key servers you can publish your keys to. Beyond that, GitHub</a> also provides a mechanism for publishing your GPG</a> public keys as well. Just go to your Settings, SSH & GPG Keys, and add your GPG Public Key. You can also publish your public key to a key server using GPG</a> directly. The following is an example: $ gpg --send-keys 41C5D2C6E5AF944C gpg: sending key 41C5D2C6E5AF944C to hkps://hkps.pool.sks-keyservers.net</code></pre> Note: The major key servers share and sync the public keys so you shouldn't have to worry about dealing with multiple key servers. Verify and Sign</h4> Once you have published your key, you need to work with people to verify that the key they have actually belongs to you. This is often done at a Key Signing Party</a> to be the most secure. However, I have known many to verify their keys over the phone, as long as the people can answer questions proving they are who they say they are. You can search for someone's public keys via their email address on the key servers using the following: $ gpg --search-keys <their email address></code></pre> After you find and import their public key, you need to verify it and sign it. To verify it you can have your own little signing party, or do it via phone if you are comfortable with that. Either way, the process of verification is the same. First ask the person for the fingerprint of their public key. Next verify that it matches the fingerprint of the public key you have for them. Remember, you can see the fingerprint of their key by doing, gpg --fingerprint <their email address></code>. Once you have verified they match, you can define your level of trust for them using the following: $ gpg --edit-keys <their email address> > trust</code></pre> For details around the different levels of trust and what they mean you can refer to the Trust in a key's owner</a>. You can also fully sign a public key by doing the following: $ gpg --edit-keys <their email address> > sign</code></pre> You can find out more about validating keys in the GPG Manual - Validating other keys on your public keyring</a>. More information is also available on exchanging keys and signing keys in the GPG Manual - Exchanging keys</a>. Confusion</h4> I know this topic can be a little confusing at times. A few parts are still confusing for me when I haven't dealt with them in a while. So, if you are confused with terms, or want a deeper understanding of the entities involved and their attributes with GPG</a>, I recommend checking out the article Anatomy of a GPG Key</a>. If you are confused about GPG</a> itself, they have decent documentation</a>. Once all of the above has been done you should be able to cryptographically verify and manage your source code with true identity representation and trust. Remote Stand-ups at UpTech 2017-07-10T00:00:00+00:00 The Problem</h3> Given that UpTech is a consultancy, our team is often pretty fragmented in terms of physical location. For example, I might be on-site at a client's office while another member is at a different client's office, and yet other members are at our office. This physical fragmentation makes it difficult to stay on top of what is going on across the organization on a day-to-day basis. Normal Solution</h3> Normally, in a startup we would have a stand-up every morning to kick-off the day. If you aren't familiar, a stand-up is a time-boxed meeting that is used in many agile methodologies to facilitate communication around individuals' daily commitments, as well as raise awareness of any challenges (a.k.a. blockers). The reason it is called a stand-up is because the thinking is that if everyone has to stand during the meeting it would help keep the meeting short. For further details on stand-ups and their intent check out the Wikipedia page</a>. Our Solution</h3> Given that our team is rarely in the same physical location at the same time, we have taken the principles and intent from the stand-up and adapted it to our situation by using a Slack</a> #standups</code> channel to share our commitments and blockers every morning at 10 am. To aid with making sure we have the stand-up right at 10am everyday, we set up a Slack reminder</a> in the #standups</code> channel. Then in the #standups</code> channel we communicate the classic stand-up information: What did I do yesterday?, What am I doing today?, Do I have any blockers? Specifically, we use the following template and generally prepare the mornings stand-up in a text buffer before pasting it into the #standups</code> channel. The template/example below is based on the Slack message format</a>. _Tue July 11, 2017 - Standup_ *Yesterday* _client_1_ • One thing I did yesterday • Another thing I did yesterday _client_2_ • Yet another thing I did yesterday *Today* _client_1_ • A thing I am going to do today • Something else I am going to do today _client_2_ • Another thing I am going to do today *Blockers* • A blocker that is currently preventing me from accomplishing something or will in the near future</code></pre> When the above is pasted into Slack</a> it looks as follows: Given that this is such a core part of our process and that I am a heavy Vim</a> user, I created the Vim Slack Format</a> plugin to provide syntax highlighting in Vim</a> for the Slack message format</a>. When using this plugin it looks as follows: Without this virtual stand-up every morning we wouldn't be able to successfully run our business. It constantly triggers discussions, questions, suggestions, etc. following the stand-ups. How you can use this?</h3> If you work for a consultancy yourself, you might find this solution and template above valuable for the same reasons we do. However, if instead you are simply on a remote team, where there aren't multiple clients you could easily modify the format to the following simpler version and get the same value out of it. _Tue July 11, 2017 - Standup_ *Yesterday* • One thing I did yesterday • Another thing I did yesterday • Yet another thing I did yesterday *Today* • A thing I am going to do today • Something else I am going to do today • Another thing I am going to do today *Blockers* • A blocker that is currently preventing me from accomplishing something or will in the near future</code></pre> At the end of the day, like any tool/process. It comes down to having an open minded collaborative team that understands the value of process and communication to really get the value out of a practice like this. Tools are never a solution for culture. RSpec Basics Screencast 2017-02-04T00:00:00+00:00 This is yet another screencast I dug up while cleaning out some of the digital clutter. This one provides a great basic walk through of outside in development despite using a sligtly older version of RSpec. The biggest difference in RSpec versions at this point is the way expections are written. In this screencast it references the old style: Catalog.should_receive(:all)</code></pre> The new style avoids implementations that require monkey patching. This now looks as follows. expect(Catalog).to receive(:all)</code></pre> Despite this small deviation this screencast is still extremly valuable and hopefully helps people get a grasp on outside in development and the basics of RSpec. You should be able to easily adjust for the difference by referencing the rspec-expectations</a> documentation. Your browser does not support the video</code> element. Please upgrade/switch to a more modern browser that does if you want to be able to view videos. </video> .m4v</a> .ogv</a> .mov</a> In this episode we reset the state of code to the point where in episode 2 we should have dropped down and fleshed out the RSpec tests for the controller. This episode covers the very basics of RSpec while we drive out a Rails controller action as part of the feature outlined in episode 2. Release Date</th> 2013-05-15</td> </tr> Size</th> 79.7 MB</td> </tr> Duration</th> 21 mins</td> </tr> Resolution</th> 1920 x 1080</td> </tr> </table> Cucumber Basics Screencast 2017-01-07T00:00:00+00:00 This is another screencast I dug up while cleaning out some of the digital clutter in my life. This one still provides a great basic walk through of how to properly define acceptance tests and use them to drive out code with Cucumber. Your browser does not support the video</code> element. Please upgrade/switch to a more modern browser that does if you want to be able to view videos. </video> .m4v</a> .ogv</a> .mov</a> In this screencast I give a basic introduction to Cucumber — what Cucumber is and how to use it. I use a real-world example of adding the list screencasts feature to The Code Breakdown Rails app. I start out writing a Cucumber feature and explaining it, then dive into using that feature to drive out the implementation. Release Date</th> 2012-12-04</td> </tr> Size</th> 33.6 MB</td> </tr> Duration</th> 16 mins</td> </tr> Resolution</th> 1920 x 1080</td> </tr> </table> Git Under the Hood Screencast 2016-12-18T00:00:00+00:00 Back in 2013 I did a series of screencasts with a good friend of mine, Brian Miller</a>. I was going through things trying to remove some of the digital clutter and noise in my life. In the process I found this screencast and realized it is still very applicable today. Therefore, I figured I might as well share it. Your browser does not support the video</code> element. Please upgrade/switch to a more modern browser that does if you want to be able to view videos. </video> .m4v</a> .ogv</a> .mov</a> In this episode, Brian and I cover the data structure behind git, what commits are really composed of, and how we think people should try and think when using Git. The hope is that this will help get people over one of the larger hurdles in the learning curve of Git. Release Date</th> 2013-07-30</td> </tr> Size</th> 189.9 MB</td> </tr> Duration</th> 16 mins</td> </tr> Resolution</th> 1920 x 1080</td> </tr> </table> Building a SegmentedViewController 2016-12-16T00:00:00+00:00 Let's say you run into a scenario where you need to build a mobile interface that has tabs at the top of the screen. If you are working on Android this isn't too much of a problem as it is natively supported and part of their Material Design</a>. If you are building an iOS app it is a bit of a different story as the native UITabBarController</code> doesn't really work at the top of the screen. Yes, there are some nasty things you can do to hack it. But, we really shouldn't be hacking things, should we? If we want to not run into crazy issues later and continue the hack, we should generally use the framework and systems as they were designed to be used. If we dig into Apple's iOS Human Interface Guidelines</a> the closest thing I can find to tabs at the top of the screen is in the UI Bars -> Navigation Bars</a> section. Looking through the images you will find the image to the left and an explanation suggesting that you use UISegmentedControl</code> in the navigation bar at the top to help flatten your information hierarchy. So, how do we actually make something like this happen? It turns out that constructing an instance of a UISegmentedControl</code>, injecting it into the UiNavigationBar</code>, and getting callbacks when the various segments are selected is pretty straight forward. var segmentedControl = UISegmentedControl(items: ["New", "Replied"]) segmentedControl?.selectedSegmentIndex = 0 segmentedControl?.addTarget(self, action: #selector(segmentSelected), for: .valueChanged) yourViewController.navigationItem.titleView = segmentedControl</code></pre> The above constructs a UISegmentedControl</code> with two segements, New, and Replied. It also selects New. Note: This is only a visual indication it doesn't trigger any callbacks. Then we call addTarget</code> to register an action to be called when the value of the segmented control changes. This is the spiked ground work for most of the components necessary. However, there is a lot more involved to have the UISegmentedControl</code> trigger changing the views. The intended approach for this is to use a concept Apple created called a Container View Controller</a>. These are basically responsible for owning and managing a collection of view controllers and their associated views. Prime examples of these are UITabBarController</code> and UINavigationController</code>. Turns out none of the existing Container Controllers</a> do what we want. So, we will have to Implement a Container View Controller</a> of our own that takes care of owning/managing the UISegmentedControl</code>, the parenting view,and managing a collection of View Controllers and their paired Views that are coupled to each segment of the UISegmentedControl</code>. Walk Through</h2> Given that we now have an understanding of the core components and concepts. Lets walk through implementing a rough initial version of our custom container view controller. Let's start by creating a class that inherits from UIViewController</code> called SegmentedViewController</code>. import UIKit open class SegmentedViewController: UIViewController { }</code></pre> Now we know that the SegmentedViewController</code> needs to manage a collection of view controllers as well as a segmented control. So, let's create stored properties for both of those. import UIKit open class SegmentedViewController: UIViewController { open var segmentViewControllers: [UIViewController] = [] open var segmentedControl: UISegmentedControl? }</code></pre> Now that we have properties to hold the managed values we need to initialize them. We do this by defining a constructor that takes viewControllers: [UIViewController]</code> and assigns them to our stored property self.segmentViewControllers</code>. Note: We also have to define the required constructor enforced by UIViewController</code>. However, we don't want to support compilation via that mechanism so we have it fatal error out. import UIKit open class SegmentedViewController: UIViewController { open var segmentViewControllers: [UIViewController] = [] open var segmentedControl: UISegmentedControl? public init(_ viewControllers: [UIViewController]) { super.init(nibName: nil, bundle: nil) self.segmentViewControllers = viewControllers } public required init(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") } }</code></pre> Given that we can now have an object that knows about a collection of view controllers to manage, we need to use those to set up our other stored property, the UISegmentedControl</code>. We can do this by using the standard viewDidLoad</code> method of the UIViewController</code> and defining a couple more methods. import UIKit open class SegmentedViewController: UIViewController { open var segmentViewControllers: [UIViewController] = [] open var segmentedControl: UISegmentedControl? public init(_ viewControllers: [UIViewController]) { super.init(nibName: nil, bundle: nil) self.segmentViewControllers = viewControllers } public required init(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") } open override func viewDidLoad() { super.viewDidLoad() self.segmentedControl = UISegmentedControl(items: segmentTitles()) self.segmentedControl?.selectedSegmentIndex = 0 self.segmentedControl?.addTarget(self, action: #selector(segmentSelected), for: .valueChanged) self.navigationItem.titleView = self.segmentedControl } fileprivate func segmentTitles() -> [String] { var segmentTitles: [String] = [] for viewController in self.segmentViewControllers { if let title = viewController.title { segmentTitles.append(title) } } return segmentTitles } open func segmentSelected(_ segmentControl: UISegmentedControl) { } }</code></pre> In the above, we added in some of the basic ground work that we covered at the beginning of the article. Specifically, we constructed the UISegemntedControl</code> and assigned it to our stored property, self.segementedControl</code>. We marked the first segment as the active one, visually, and we added a target called segmentSelected</code> when the value of the segmented control changes. The segmentTitles</code> method is a simply helper method that extracts the standard title</code> property from the injected view controllers. This allows us to easily use them when constructing the UISegmentedControl</code>. Now we need to add the ability for it to display the first view controller, as we already configured the self.segmentedControl</code> telling it that the first one is selected. import UIKit open class SegmentedViewController: UIViewController { open var segmentViewControllers: [UIViewController] = [] open var segmentedControl: UISegmentedControl? public init(_ viewControllers: [UIViewController]) { super.init(nibName: nil, bundle: nil) self.segmentViewControllers = viewControllers } public required init(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") } open override func viewDidLoad() { super.viewDidLoad() self.segmentedControl = UISegmentedControl(items: segmentTitles()) self.segmentedControl?.selectedSegmentIndex = 0 self.segmentedControl?.addTarget(self, action: #selector(segmentSelected), for: .valueChanged) displaySegmentViewController(0) self.navigationItem.titleView = self.segmentedControl } fileprivate func segmentTitles() -> [String] { var segmentTitles: [String] = [] for viewController in self.segmentViewControllers { if let title = viewController.title { segmentTitles.append(title) } } return segmentTitles } open func segmentSelected(_ segmentControl: UISegmentedControl) { } fileprivate func displaySegmentViewController(_ segmentViewControllerIndex: Int) { let viewController = self.segmentViewControllers[segmentViewControllerIndex] self.addChildViewController(viewController) self.view.addSubview(viewController.view) viewController.view.translatesAutoresizingMaskIntoConstraints = false NSLayoutConstraint(item: viewController.view, attribute: .left, relatedBy: .equal, toItem: self.view, attribute: .left, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .right, relatedBy: .equal, toItem: self.view, attribute: .right, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .top, relatedBy: .equal, toItem: self.view, attribute: .top, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .bottom, relatedBy: .equal, toItem: self.view, attribute: .bottom, multiplier: 1.0, constant: 0.0).isActive = true viewController.didMove(toParentViewController: self) } }</code></pre> In order to accomplish this, we added a displaySegmentViewController</code> method that given a segmentViewControllerIndex</code> appropriately displays and manages the lifecycle of the child view controller and its view. Now we need to add support so that when a segement is selected it appropriately displays the selected segment's view and hides the previously selected segment's view. import UIKit open class SegmentedViewController: UIViewController { open var segmentViewControllers: [UIViewController] = [] open var segmentedControl: UISegmentedControl? fileprivate var lastDisplayedSegmentViewControllerIndex: Int? public init(_ viewControllers: [UIViewController]) { super.init(nibName: nil, bundle: nil) self.segmentViewControllers = viewControllers } public required init(coder aDecoder: NSCoder) { fatalError("init(coder:) has not been implemented") } open override func viewDidLoad() { super.viewDidLoad() self.segmentedControl = UISegmentedControl(items: segmentTitles()) self.segmentedControl?.selectedSegmentIndex = 0 self.segmentedControl?.addTarget(self, action: #selector(segmentSelected), for: .valueChanged) displaySegmentViewController(0) self.navigationItem.titleView = self.segmentedControl } fileprivate func segmentTitles() -> [String] { var segmentTitles: [String] = [] for viewController in self.segmentViewControllers { if let title = viewController.title { segmentTitles.append(title) } } return segmentTitles } open func segmentSelected(_ segmentControl: UISegmentedControl) { var lastDisplayedIndex: Int? = nil if let i = self.lastDisplayedSegmentViewControllerIndex { lastDisplayedIndex = i } displaySegmentViewController(segmentControl.selectedSegmentIndex) if let j = lastDisplayedIndex { hideSegmentViewController(j) } } fileprivate func displaySegmentViewController(_ segmentViewControllerIndex: Int) { let viewController = self.segmentViewControllers[segmentViewControllerIndex] self.addChildViewController(viewController) self.view.addSubview(viewController.view) viewController.view.translatesAutoresizingMaskIntoConstraints = false NSLayoutConstraint(item: viewController.view, attribute: .left, relatedBy: .equal, toItem: self.view, attribute: .left, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .right, relatedBy: .equal, toItem: self.view, attribute: .right, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .top, relatedBy: .equal, toItem: self.view, attribute: .top, multiplier: 1.0, constant: 0.0).isActive = true NSLayoutConstraint(item: viewController.view, attribute: .bottom, relatedBy: .equal, toItem: self.view, attribute: .bottom, multiplier: 1.0, constant: 0.0).isActive = true viewController.didMove(toParentViewController: self) self.lastDisplayedSegmentViewControllerIndex = segmentedControl?.selectedSegmentIndex } fileprivate func hideSegmentViewController(_ segmentViewControllerIndex: Int) { let viewController = self.segmentViewControllers[segmentViewControllerIndex] viewController.willMove(toParentViewController: nil) viewController.view.removeFromSuperview() viewController.removeFromParentViewController() } }</code></pre> To do this we added a fileprivate stored property called lastDisplayedSegmentViewControllerIndex</code> to track the previously selected segment. This was useful when we fleshed out the segmentSelected</code> method. We also updated the displaySegmentViewController</code> method to update lastDisplayedSegmentViewControllerIndex</code> with a new index after it has displayed the segment's view. The details on the required lifecycle management of the child view controllers that we implemented in the displaySegementViewController</code> and hideSegmentViewConroller</code> were provided in Implement a Container View Controller</a>. This completes our initial implementation of our SegmentedViewController</code>. Usage</h2> Let's look at how we would use this thing now. The following is a simple AppDelegate</code> that would use the SegemntedViewController</code>. import UIKit @UIApplicationMain class AppDelegate: UIResponder, UIApplicationDelegate { var window: UIWindow? = UIWindow(frame: UIScreen.main.bounds) func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool { let vcOrange = UIViewController() vcOrange.title = "Orange" vcOrange.view.backgroundColor = .orange let vcYellow = UIViewController() vcYellow.title = "Yellow" vcYellow.view.backgroundColor = .yellow let segmentedViewController = SegmentedViewController([vcOrange, vcYellow]) window!.rootViewController = UINavigationController(rootViewController: segmentedViewController) window!.makeKeyAndVisible() return true } }</code></pre> That's It</h2> All in all it isn't too bad. I hope you found this valuable and if you have any questions don't hesitate to contact me. This is just a little more complex than I would want to have to deal with in an app code base. So, I will probably break this out into a framework soon. The UpDesk UpWrite 2013-12-11T00:00:00+00:00 Backstory</h2> Over the last few years I have been paying more and more attention to ergonamics in my workspace. It all began when I started having issues with my wrists and given I am a software developer I need my hands to work properly. This sent me down a path of exploring ergonomic keyboards and standing desks. I started trying out the cheaper ergonomic keyboards and never had any luck. Enter the Kinesis Advantage Pro, I finally decided to drop a chunk of change on it and I really haven't looked back. My next exploration will be the ErgoDox which I recently ordered all the parts for. A little bit before I switched to the Kinesis Advantage Pro I switched to a fixed height standing desk which I bought from Ikea for approx. $150. I have been using the fixed height standing desk for over a year now and I love it. However, the more research I do around the topic the more I have found that it isn't good to stand all day, which is what I currently do. Therefore, I started looking into motorized standing/sitting desks. After a lot of research I finally decided on the UpDesk UpWrite</a>. It is at the lower end of the price range, $1,149.00, for motorized standing/sitting desks but it has the specs of a lot of the more expensive alternatives. It also has the added benefit that it is a dry erase/wet erase surface so you can scribble notes on it while working. The Setup</h2> Putting it Together</h3> My UpDesk UpWrite</a> arrived tonight and it was actually very straightforward in terms of physical setup. The included instruction sheet explained things very well. It is actually much simpler in my opinion than most Ikea furniture. It probably took me 25 mins to get it put together. I was extremely impressed with the quality of packaging and the quality of the parts them selves. It seems to be an extremely solid piece of furniture that will last a long time. Initializing and Calibrating</h3> Once I physically setup the desk I tried to initialize it and callibrate it. Initializing the desk was very straightforward on their instruction sheet and worked like a champ. Calibrating it on the other hand did not work following the instructions they provide. The instructions for calibration that were provided are as follows: At any height, measure the distance from the floor to the top of the desk.</li> Press both the "UP" and "DOWN" arrows at the same time until the display shows three bars (---).</li> Release and immediately set the correct height by pressing either the "UP" or "DOWN" arrow until the correct height is displayed.</li> </ul> I tried the above steps numerous times and every time I got to the third step and pressed the "UP" or "DOWN" arrows to adjust the displayed height it would also move the desk up or down which made no sense if the goal of the process was to callibrate the height. So, it took a little playing around but I finally figured out how to properly callibrate it. The directions to calibrate it should really be as follows: At any height, measure the distance from the floor to the top of the desk.</li> Press and hold both the "UP" and "DOWN" arrows at the same time, the display will immediately display three bars (---), continue to hold the "UP" and "DOWN" arrows until the display flashes from (---) to the displayed height.</li> Release and immediately set the display to height you previously measured in the first step by pressing either the "UP" or "DOWN" arrow until the correct height is displayed.</li> </ul> Dialing in Presets</h3> Once I had it all calibrated I followed the rest of the instructions to set the presets to heights that I like for standing and sitting. All worked great. Conclusion</h2> So far the UpDesk UpWrite</a> seems like a great desk. It was a little frustrating that the directions weren't clear enough, or rather just plain wrong, for how to calibrate it. However, I am willing to look past that small hickup and I hope this blog aids others in getting their desks calibrated quicker than I did. Shrinking Vagrant Linux Boxes 2013-10-29T00:00:00+00:00 So, lately I have been using Virtual Machines via Vagrant</a> a siginificant amount as we are now using them to host our development environments and dependencies for all the development work we do. We have been using Puppet</a> to automate the configuration of our environments and to make building our boxes easier. Once you have provisioned a VM and you have it all dialed, you will most likely want to create a Vagrant Box</a> from that VM and host it up on S3 so that new developers can get started simply by installing Vagrant</a>, cloning your repository and running: $ vagrant up</code></pre> This actually works great. However, I found that the VM disks both for the Vagrant VMware Provider</a> and the VirtualBox Provider</a> grow dynamically, but they never shrink. So, when I was going through the process of building the VM and installing all the dependency software, etc. it was bloating the base machine. This was the case even though I removed the temporary cruft from the filesystem on the guest. With a little research and some trial and error I found that you can shrink these disks as long as they are ext3 or ext4 file systems with the help of the following command: $ vagrant ssh $ sudo dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile</code></pre> What the above command is doing is simply writing zero bytes to the wipefile in chunks of 1024 bytes until there is no disk space left in your VM's disk. Then it is removing the wipefile. This basically leaves all those excess bytes zero'd out. This is necessary because the shrink/compaction tools provided by either VMWare</a> or VirtualBox</a> both have no way of identifying space they can free up in the disks unless they are zero'd out. Before shrinking/compacting you should always make sure to exit the guest machine using the following commands: $ exit $ vagrant halt</code></pre> With VMware</a> you can shrink the vmdk disk by doing the following: $ vmware-vdiskmanager -d /path/to/main.vmdk $ vmware-vdiskmanager -k /path/to/main.vmdk</code></pre> I discovered the above commands in the Vagrant VMware documentation, http://docs.vagrantup.com/v2/vmware/boxes.html</a> With VirtualBox</a> the only way I was able to shrink the disk image was to clone it to a smaller copy using the following command: $ VboxManage clonehd name-of-original-vm.vdi name-of-clone-vm.vdi</code></pre> Once you have cloned the vdi you can then import it into the VM through VirtualBox and get rid of the original vdi. That is how you "Shrink Vagrant Linux Boxes!" AT&T Samsung Galaxy S3 with Google Voice 2013-01-06T00:00:00+00:00 Hi All, It seems there is a little hickup with the instructions google provides you with to "Activate Google voicemail no this phone". That is at least for the instructions for AT&T while trying to get Google Voice setup on an AT&T Galaxy S3. The instructions Google provides say to enter the following exactly as written as if dialing someone. *004*1<your 10 digit phone number>#</code></pre> Every time I tried the above instructions my phone would return an error code. I got so frustrated I even got online with AT&T tech support so that they could verify the state of things as I was trying various other commands that I found on the web. None, of the ones that worked allowed me to setup Google Voice as my voicemail provider appropriately. After being online with AT&T tech support for a while. I tried simply prepending another * character to the entire thing. Guess what, magically it worked. I even had AT&T tech support verify that it made the proper configuration change on my account. So the correct command that Google should be telling people to enter exactly as written is really the following: **004*1<your 10 digit phone number>#</code></pre> To disable voicemail forwarding I used the following MMI code. ##004#</code></pre> Anyways, hope this blog post helps some other people struggling with this out there. I know when I was searching I couldn't find any solution though I did see a number of message boards with people looking for an answer to this problem. If I were a better person I would probably go back and update all those message boards with my solution. I just can't bring myself to do it. RVM Installed Ruby (SSL certificate verify failed) 2012-09-07T00:00:00+00:00 Recently a few of our developers were having problems getting SSL certificate verify failures in Ruby. The error was the following: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `connect' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `block in connect' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/timeout.rb:54:in `timeout' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/timeout.rb:99:in `timeout' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `connect' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:755:in `do_start' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:744:in `start' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:306:in `open_http' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:201:in `catch' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:677:in `open' from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:33:in `open'</code></pre> The error was reproducable by running the following Ruby snippet in an irb instance of the RVM installed Ruby version. require 'open-uri' open 'https://google.com'</code></pre> I spent a good amount of time digging into things and trying to figure out exactly what the issue was. I came to the conclusion that they either had an out of date Root CA Certificate Bundle or SSL couldn't find it for some reason. So, I went down the path of trying to import cacert.pem from http://curl.haxx.se/ca/cacert.pem</a> (cURL's website). I went down this path because I vaguely remember doing this at some point in the past to solve the Twitter API SSL Verify fail issue when they changed their certs. I ran into a huge amount of difficulty trying to get the thing imported or converted into a format that Mac OS X Keychain Access would be happy with importing. I tried pem, crt, pck12, etc. Nothing I did seemed to make a difference. This is when I saw a post of someone talking about SSL being installed inside of RVM and I looked at the latest output from rvm requirements</code>. I quickly noticed that it now says to install SSL via the following command: rvm pkg install openssl</code></pre> Come to find out if you have run the above command before installing your Ruby version it will be using the RVM packaged openssl instead of the system openssl. That also means that the Root CA Certificate Bundle is different, or in this case, missing completely. After discovering this and doing a little googling I found the solution to be as simple as the following command: curl http://curl.haxx.se/ca/cacert.pem -o ~/.rvm/usr/ssl/cert.pem</code></pre> I found this solution on the following blog post http://fredwu.me/post/28834446907/fix-openssl-error-on-mountain-lion-and-rvm</a>. Hopefully, writing another blog post on the subject and linking to this one will help it gain its ranks in google search so that other people can find it quicker. After running the above command you should now be able to restart your irb instance and re-run the test snippet described at the beginning of this post. This time you should get output looking similar to the following: => #<File:/var/folders/j_/9nxyjmd10js8g1sjx446vrsm0000gn/T/open-uri20120907-36935-4i4y64> </code></pre> Just like that your SSL woes are gone. Sublime Text 2 Guard Plugin Updates 2012-08-07T00:00:00+00:00 I woke up early this morning and thought it was time for a little sublime_guard</a> love. I went through and tackled all of the syntax highlighting issues. This included the following issues (#27</a>, #30</a>, #31</a>, #38</a>, #43</a>, #29</a>). I also cleaned up a number of tickets that were lying around and had either been completed already or were tickets that were invalid. I also started a branch for adding python3 support to the plugin. The branch is named python3_support. I would love it if anyone out there in the ether would grab that branch, test it out, and give me some feedback. Anyways, I hope you guys enjoy the updates. As usual if you installed from Package Control just restart Sublime Text 2. If you installed manually then just pull the latest version down with Git. See you all next time. Ruby and Rails Mountain Lion Hiccups 2012-07-27T00:00:00+00:00 So when Mac OS X Mountain Lion was released a few days ago I jumped on upgrading and got switched over as soon as I could. My mentality for upgrading is that I would much rather deal with the pain of upgrading in smaller chunks when needed rather than waiting. Plus, I just like to be on the latest and greatest in terms of the tools I use for my job. Anyways, below is going to be a listing of some of the hiccups that either I or other developers on my team have run into in terms of their Ruby and Rails development environment setups. Git</h2> Some of the devs on my team had Git installed via the git-osx-installer</a>. These developers seemed to lose their Git installs when they upgraded. I was using a Homebrew</a> install of Git and it worked just fine for me after upgrading. So be aware of what your situation is and resort to brew because it is awesome. XCode 4.4</h2> In order to build gems with the native exensions and just to have the basic development tools around I installed the latest XCode 4.4. Once installed I went to the preferences menu, in the download tab, and chose to install the Command Line Tools because that is really what I was looking for more than anything. I have read on other posts online that you can download the Command Line Tools alone and install just them. I personally chose to install XCode as I do use XCode and the native tools to work on iPhone/iPad app development as well. gcc-4.2</h2> The latest XCode now comes with clang/llvm which is a new C/C++/Objective-C compiler. The problem is that the Ruby language isn't fully compatible with clang and llvm. Therefore, you need to install GCC 4.2 using Homebrew. GCC 4.2 is the latest version of the GCC compiler that apple provided before switching to clang/llvm. Anyways, you can install GCC 4.2 by doing the following with brew: brew tap homebrew/dupes brew install apple-gcc42</code></pre> The above will install the apple gcc-4.2 in a side by side fashion with XCode. This means that it won't mess up your existing XCode install like other solutions. Once you install gcc-4.2 then you can install/reinstall your ruby version inside of RVM as needed. Apache 2 (Web Sharing)</h2> In the upgrade it seems that they have revamped the Sharing preference pane. In the process of this it seems that Apple has decided to remove the Web sharing component of this preference pane. Don't worry though. Apache is still there and running. You just can't start it and stop it from the preferences pane anymore. You can use the following commands instead. sudo apachectl stop sudo apachectl start sudo apachectl restart</code></pre> Note: When the upgrade happened it nuked all my /etc/apache2/users/</code> configs and all my other apache configs. So, if you have any custom apache configs as I did you should probably back them up before upgrading. X11</h2> It seems that another thing that was changed with the upgrade to Mac OS X Mountain Lion was that they removed the X11 that I had previously installed and seemed to no longer provide it. Thats ok though because XQuartz 2.7.2 is what you want and it is available freely at http://xquartz.macosforge.org/landing/</a>. Just install that bad boy and follow the on screen instructions to reboot when it is done and you should be good to go in terms of X11. ImageMagick</h2> Before upgrading I had ImageMagick installed via Homebrew. After, the upgrade was finished my ImageMagick was exiting with failures about some dylib. To solve this problem I simply ran the following and I was back in running order with respect to ImageMagick: brew upgrade imagemagick</code></pre>Pygments</h2> I use Pygments to handle syntax highlighting in my jekyll sites and it seemed to not be working after the upgrade. I believe this is because it is now using Python 2.7.2. To resolve this issue I just re-installed Pygments using the following: sudo easy_install Pygments</code></pre> Sublime Text 2 Guard Plugin Updates 2012-07-25T00:00:00+00:00 It has been a while since I have provided an update on my Sublime Text 2 Guard Plugin</a> and I just made some changes so I figured I would make a quick post. The project has recently had a number of contributions which I will list in the changes below. Fixed default Linux key-mappings (#52</a>)</li> Fixed default Windows key-mappings (#52</a>)</li> Added Run all Tests & Show Guard Output, Thanks @whitequark</a> (#44</a>, #53</a>)</li> Made it work without Bundler, Thanks @brandonblack</a> (#51</a>, #50</a>)</li> Clear output when predefined text found, Thanks @diogomafra</a> (#48</a>)</li> Added support for rbenv, Thanks @chrislerum</a> (#45</a>)</li> </ul> I have been completely stoked with the growing usage of this plugin as well as the abundance of contributions people have been making to the plugin. In this update there are a number of people that need to be called out with special thanks. @diogomafra</a> provided a Clear output when predefined text found feature. I am sure many users will find this feature a nice addition to smoothing out their workflow. Thank you for your contribution @diogomafra</a>. @chrislerum</a> provided rbenv support. I know that this feature will be huge for a number of the users as this was a highly requestd feature. Thank you so much for your contribution @chrislerum</a>. @brandonblack</a> provided the ability to function without bundler. This will allow users who aren't using bundler to be able to use this plugin. I think this is a great addition as the more people we can get using this plugin the more feedback we can get. Hence, the better it will become. Thanks for your contribution @brandonblack</a>. @whitequark</a> provided the basis for the Run all Tests & Show Guard Output feature. I know a number of developers who have been waiting for something like this to fit their personal workflow. Thank you very much for your contribution @whitequark</a>. A big thanks goes out to all the above listed contributors and all of our prior contributors. Thank you all for helping make this a solid very useful plugin for the development community. As usual please make issues for any bugs or feature requsets you have at Sublime Text 2 Guard Plugin</a>. A New Rails Tagging Plugin 2012-07-22T00:00:00+00:00 A little over 3 months ago I had the need for a good tagging plugin for Rails. I tried out a number of the popular ones but they just weren't up to my standards in terms of their APIs and their functionality. To accomplish the user experience goals that had been set forth for the project I would have had to hack the existing Rails tagging plugins to pieces. So after some discussion about the Rails plugins with my team. We decided that I should just start my own Rails tagging plugin. Anyways, I started developing Tagalong</a>, my Rails tagging plugin, immediately. The library is now 3 months old and I figured I should let people know that it exists and provide a breakdown of its features. Tagalong</a> has two major concepts that a developer needs to understand before being able to use it. The first concept is a Tagger. A Tagger is simply an object that can perform the act of tagging other objects (Taggables). As I am sure you have guessed the second concept is a Taggable. A Taggable is simply an object in the system that can be tagged. The Tagalong</a> Rails plugin is built entirely around these two concepts a Tagger and a Taggable. This architecture decision is one of the major differences between this Rails tagging plugin and the other tagging plugins out there. A brief breakdown of the plugin is as follows. clean easy to understand API</li> does NOT require saving of the model when tagging/untagging</li> keeps history of tags Taggers have used</li> allows defining multiple Taggers and Taggables</li> tracks number of times tags are used</li> returns tag lists in alphabetical order</li> </ul> The above are simply a few brief points. For details on all its features and how to use it please refer to the GitHub project http://github.com/uptech/tagalong</a>. OctopusCI now has job rerun 2012-07-22T00:00:00+00:00 I recently released v0.3.12 of my CI server OctopusCI</a>. This release provided a feature which people have been asking for a while now. I am glad to tell you all that it is finally here. OctopusCI</a> now has the capability of re-running jobs simply by clicking a link. Sublime Text 2 Guard Plugin Updates 2012-04-15T00:00:00+00:00 Well, I am back with some more updates on my Sublime Text 2 Guard Plugin</a>. This is a smaller release than the last as I have been very busy. Despite its small stature it is an exciting one as we have another contribution to honor. The following are the recent updates: Fixed comma, single quote, question mark color issues (#25</a>)</li> Fixed small color issue caused by !</code> char (#26</a>)</li> Better console colors, change 0 failures</code> from red to green (Thanks @lidanh</a>)</li> </ul> A special thanks goes out to @lidanh</a> for the code contribution. The new colors are much nicer and the 0 failures</code> being green instead of red makes it easier to detect success vs failure. Thanks also goes out to all the users of the Sublime Text 2 Guard Plugin</a>. Especially, the users that have been submitting bug reports and feature requests. This plugin wouldn't exist without the direction and feedback from you all. Please continue to provide feedback via Sublime Text 2 Guard Plugin issues</a> on our GitHub page. My zsh Setup 2012-04-13T00:00:00+00:00 Relatively recently I switched to ZSH for my shell. The primary reason I switched was that I had more flexibility in prompts as I can setup a right prompt and a left prompt. I also like the completion interfacing that ZSH provides. Anyways, I finally got around to generalizing my ZSH setup and making it available on GitHub as a backup for myself so that I can easily set it up again in the future, as well as a means of sharing my setup with all of you. It is available on GitHub at http://github.com/drewdeponte/dotzsh</a> with instructions on how to install it using git. My VIM Setup 2012-04-11T00:00:00+00:00 So, as some of you may know I pulled my old .vimrc out of hiding and started playing around with vim again to see if I would potentially want to switch back to it from Sublime Text 2. The reason being that I still fealt I was missing something that I had in vim (the quick keyboard navigation and general flexibility). Anyways, I pulled my .vimrc out and added the basics of what I think I am going to need for it to be an acceptable replacement for Sublime Text 2. This includes fuzzy file search, tab completion, etc. Basically all the things necessary for it to be a competitor to Sublime Text 2 in my mind. It isn't perferct yet but it is a start. Note: The only reason I am doing this is because I already knew VIM so I don't have a learning curve to really explore this. In my eyes Sublime Text 2 and VIM are pretty damn close in terms of functionality but Sublime Text 2 provides a much smaller learning curve for the majority of people. This does NOT mean that I am giving up on Sublime Text 2 or that I am going to stop supporting my Sublime Text 2 plugins. If it comes to that at some point in the future I will find a new mantainer for them and let you all know. Anyways, I have made my vim setup available on github via http://github.com/cyphactor/dotvim</a>. The README for the repository provides instructions on how to install using git. Sublime Text 2 Guard Plugin Updates 2012-02-28T00:00:00+00:00 Well, I got some more updates for you all. My Sublime Text 2 Guard Plugin has recently been recognized as an essential Sublime Text 2 plugin on a nettuts+ blog post. Thanks to this publicity and the exposure via Package Control, Twitter, GitHub, and Google I have gotten some decent feedback, especially around my recent addition of colored output. Thanks to all that feedback I have pushed out the following improvements to the Sublime Text 2 Guard Plugin: Support for directory paths with spaces (#19</a>, Thanks @ffmike</a>)</li> Fixed dash based coloring issue (#16</a>)</li> Fixed single quote based coloring issue (#17</a>)</li> Fixed question mark based coloring issue (#20</a>)</li> Fixed period based coloring issue (#21</a>)</li> Fixed success dot section identification & coloring issue (#22</a>)</li> Fixed a second single quote based coloring issue (#23</a>)</li> Added command relativity (#24</a>)</li> </ul> The above break down of improvements include primarily a bunch of coloring fixes, command relativity, and one very important issue that deserves some recognition, issue #19</a> by @ffmike</a>. This is the Sublime Text 2 Guard Plugin's first contribution. Much thanks goes out to @ffmike</a> for being our first contributor. As usual if any of you have identified bugs or features that you would like added please submit them on the GitHub Issues page for the Sublime Text 2 Guard Plugin</a>. Sublime Text 2 Guard Plugin in Color 2012-02-22T00:00:00+00:00 Hey all, I know there are a number of you who have been eagerly awaiting this moment. I sure have been. I am proud to announce the first release of the Sublime Text 2 Guard Plugin</a> that provides colored output. Basically, it ended up that I had to define a language file to identify the entities in the output and then define a theme to color those identified entities. Once, I did that I simply programatically set the color_scheme and syntax options for the output panel and like magic colors worked. Given the complexity of the language definition and the limited test cases I had when developing I am sure there are going to be some bugs to work out. So, as usual please create issues if you find any of these issues</a>. Sublime Text 2 Guard Plugin Updates 2012-02-09T00:00:00+00:00 Hey all, My Sublime Text 2 Guard plugin has been out for a couple weeks now and it has definitely gotten some more usage under its belt. Thanks to everyone for your usage and feedback. As a result of the feedback and user generated issues I have released a new version to tighten it up a bit both from a user experience stand point as well as a stability standpoint. This release includes the following: Now installable via Package Control</a></li> Fixed Guard won't start when installled via Package Control (issue #11</a>)</li> Added auto show output feature (issue #12</a>)</li> Replaced #!/bin/sh</code> with #!/bin/bash</code> in internal scripts</li> </ul> For info on how to get the latest version or how to report bugs, feature requsts, etc. Please check out the GitHub page for the sublime_guard</a> project. Sublime Text 2 Guard Plugin Updates 2012-02-01T00:00:00+00:00 Hey all, I was having some difficulties with a couple bugs in the Sublime Text 2 Guard plugin I have been working on. So, I decided I would drop Jon Skinner, developer of Sublime Text 2, an e-mail and see if he knew about the issues I was having and hopefully of solutions/work arounds for them. Jon, was very helpful in terms of better understanding the issues as well as suggesting some work arounds. Anyways, given the input from Jon I took another look at the bugs/features of the Guard plugin and decided to tackle them tonight. The following items show the progress that I made tonight with respect to the Sublime Text 2 Guard plugin: Enabled native Sublime Text 2 word wrap in the Guard output view.</li> Guard now exits properly when Sublime Text 2 exits/dies.</li> Added default key bindings to show Guard output view (super+shift+c)</li> Submitted a pull request to get the Guard plugin added to the Package Control</a> repository list.</li> </ul> As usual I am looking for feedback issues (feature requests, bug fixes, etc.) You can find the project as sublime_guard</a> on GitHub. As of tonight I am down to two feature requests. So, please create bugs in the issues if you run into any so that I can work on fixing them sooner rather than later. Sublime Text 2 Guard Plugin Updates 2012-01-27T00:00:00+00:00 Just dropping a quick note to all that follow my blog. Last night I made a numeber updates to the Guard Plugin</a> for Sublime Text 2</a>. Most of these were in direct response to feedback that I got from users after making the initial release on my blog, I think, two days ago. The modifications I made are enumarted below: The biggest addition is RVM detection and integration</code>. The plugin is now intelligent enough to detect if you have RVM installed and detect if you are using a Project specific .rvmrc and of course respond by calling Guard appropriately depending on the case.</li> Another relatively small fix is a vertical only auto-scroll</code>. Before, if Guard output went to the right past the width of the pane it was auto-scrolling to the right and then bouncing back to the left at the next line. Now, it always scrolls down the left most column of the output basically making it auto-scroll vertically.</li> Third I added some output preprocessing that strips out the Terminal Color Codes</code> so that the output is a bit cleaner.</li> I also added a lot of descriptive output and error handling and output</code> to let the user better know what is failing so they might be able to debug the situation a bit better.</li> </ul> As always if you have any feedback (feature requests, bug reports, etc.) you can simply create an issue for the project at the projects GitHub Issues Page</a>. If you are new to the Guard Plugin</a> for Sublime Text 2</a> you can find installation instructions and document on the GitHub Page</a>. For instructions on upgrading please checkout the GitHub Page</a>. Sublime Text 2 Guard Plugin Debut 2012-01-25T00:00:00+00:00 Hi all. I know it has been a while. Please forgive me. My life has been a little crazy. I am still in the process of buliding my house. I also recently became engaged so I am trying to deal with wedding planning. Beyond that I am still trying to get all my development work done for RealPractice, my side company, and all of my open source projects. Anyways, I needed a break from serious projects and decided it was time for a small fun project that would potentially help me develop the more serious projects more efficiently. A while back I started using a new text editor called Sublime Text 2</a>. Basically it is what TextMate 2 promised (and failed) to be and more. So, since I have been using this for a while and I have been developing Ruby and Rails apps lately using BDD and TDD I decided it was time to remove the annoyance of having to switch to the terminal after each little development iteration to check Guard's output to see why my tests are failing. Wow that was a run on sentence. Oh, well it is 2 am and I just finished the plugin so bare with me. Anyways. Therefore, I decided it was the perfect opportunity to build a Sublime Text 2</a> plugin that integrates Guard into it. It is available with its documentation on GitHub at http://github.com/cyphactor/sublime_guard</a>. Please don't hesitate to go grab it, use it, extend it, contribute back, report bugs, or throw me new feature requests. Archaic - Archived Chats and Dev Helper 2010-12-13T00:00:00+00:00 So for those of that don't know, I lead of team of software developers. For a long time now I have noticed a number of gaps in the industry with respect to software development and team collaboration. One area that I always felt was missing some love was group chat. Sure, there are a large number of different chat protocols and applications that implement them all. However, they generally miss an archived history that has full text searching. Beyond the group chat is prime location to distribute push notifications when Git repositories that the team is working with have changes pushed to them. Because of these two issues and number of other that I really don't want to get into at this point I have started a new open source project called "Archaic"</a>. Archaic</a> is a Rails application that provides archiving of IRC channel chats with a full text search interface as well as general developer helpers. It is composed of two pieces, an IRC Bot and the Rails application. Technically they both share the rails environment as the IRC Bot should be run with rails runner. You can checkout the project at the following: http://github.com/cyphactor/archaic</a>. I plan to keep the master branch functional so feel free to use the master branch. It currently has the following features. IRC Chat Archiving</li> Full Text Chat Archive Searching with Context (to expand context simply click on resulting match)</li> Week number helper (when anyone in IRC channel types "weeknum" it replies with current week number)</li> Ticket number helper (when anyone in IRC channel types "#\d+" it replies with link to that ticket)</li> </ul> This is the initial introduction to this project and I am sure it has numerous directions that it can and will grow into. I very much welcome your thoughts, suggestions, feature requests, bug reports, testing, etc. I believe that having a good central place of communication for a team even if they are working in the same office helps at least as a mechanism for keeping your devs notified about repo pushes close to real time. Anyways, enjoy! Rails friendly_id Plugin 2010-08-26T00:00:00+00:00 Hey all. I recently discovered the friendly_id plugin for Rails at a Orange Country Ruby Group meetup. It basically provides a bunch of awesome functionality with respect to permalinking and slugging. Anyways, I very much recommend that you all check it out and play around with it. You can get to the projects GitHub page here</a> and its documentation here</a>. Ruby Class vs. Instance Diagram 2010-08-13T00:00:00+00:00 One of the many things that I have changed on my new team is the language. They were originally all PHP developers and my boss and I who have been using Ruby and Rails for years now decided to develop the new products in Ruby and Rails where fitting. Anyways, as a part of the process I have been educating (and being educated) by my engineers on Ruby and Rails as they go through the learning process. One of the topics that seems to have some confusion around it is Class vs. Instance variables with respect to classes and objects. Because this confusion I spent some time with one of the engineers to put together a quick diagram to try and help clarify what is actually happening with respect to Class vs. Instance variables and methods. You can find the diagram here</a>. Git, finding abandoned commits... 2010-07-30T00:00:00+00:00 So, as many of you may know I recently took a position heading up a new software engineering team that is responsible for producing the new products. One of my first actions when I came on board was switching my team over from SVN to Git. One of the engineers on the team abandoned a branch in one of our Git repositories because he didn't think he would need it anymore. Soon, after he realized that he wanted something from that branch but now he couldn't see the brach anymore in 'gitk --all' or any of the other tools because he abandoned it. But, it has to still be there right, it is Git. I knew there was a way to do this in Git and I have done it in the past but I couldn't remember how. After I explained a bit more to him about the internals of Git and he went off and did some exploration he came back to me with his found solution which follows: git fsck --unreachable| cut -f3 -d' '|xargs git rev-list --header|less</code></pre> I know there is some other way to find abandoned nodes in Git but I still can't remember. Anyways, just figured I would share this method in the hopes that it would help someone out there. It may not be exactly what you want with respect to the 'cut' or 'xargs' but I will let you all play with it a bit to get exactly what you want. jQuery Comparative Histogram Plugin 2009-10-31T00:00:00+00:00 I recently wrote a jQuery Plugin that I am calling jquery_comphist. It is a jQuery Plugin that draws a Comparative Histogram of a provided population divided into two major groups with any number of shared sub-groups. One of the most common uses for such a feature is when you are trying to display demographics data on some population. Anyways, as usual the code is available here (jquery.comphist.js</a>, jquery.comphist.css</a>) and you can see an example of it below. With the jQuery Comparative Histogram (CompHist) Plugin you can easily represent populations and information about them. A prime usage exampe of this is demographics. All the styling is done via CSS. Hence the style is easily modified. It does the calculations for you and it allows for non listed group counts via a non_demo_count</code> data option. The above demo was produced by defining a few code snippets. The first snippet is a snippet of HTML that includes the jQuery Comparative Histogram javascript source file and CSS style sheet. Note: The jQuery Comparative Histogram javascript source file needs to come after the standard jQuery javascript source file. An example of this snippet is as follows: <script type="text/javascript" src="../jquery.comphist.js"></script> <link rel="stylesheet" type="text/css" href="../stylesheets/jquery.comphist.css"></code></pre> Note: In the above snippet the href values are likely to be different in your case. The second code snippet is simply a single line of HTML where the div that is to be the Comparative Histogram. It is key that this div has an id that is easily referenced and a set height and width. An example of this snippet is as follows: <div id="compplaceholder" style="width: 375px; height: 200px;"></div></code></pre> The third code snippet needed is the javascript that defines the data, creates the Comparative Histogram in the specified div, and handles the hover functionality. An example of this is seen below. <script type="text/javascript"> $(function () { function showCompHistTooltip(x, y, contents) { console.log("called my shit"); $('<div id="comphist1tooltip">' + contents + '</div>').css({ position: 'absolute', display: 'none', top: y - 20, left: x - 30, border: '1px solid #fdd', padding: '2px', 'background-color': '#fee', opacity: 1 }).appendTo("body").fadeIn(200); } var comphist_data = { population_label: 'Active Fans This Week', group_1_label: 'Male', group_2_label: 'Female', subgroups: [ { label: '13-17', group_1_count: 3, group_2_count: 16 }, { label: '18-24', group_1_count: 7, group_2_count: 27 }, { label: '25-34', group_1_count: 3, group_2_count: 29 }, { label: '35-44', group_1_count: 2, group_2_count: 9 }, { label: '45-54', group_1_count: 0, group_2_count: 4 }, { label: '55+', group_1_count: 0, group_2_count: 0 } ], non_demo_count: 0 }; $("#compplaceholder").comphist({}, comphist_data); $("#compplaceholder").bind("barover", function (e, perc, pos) { $("#comphist1tooltip").remove(); showCompHistTooltip(pos.left, pos.top, perc); }); $("#compplaceholder").bind("barout", function (e, perc, pos) { $("#comphist1tooltip").remove(); }); }); </script></code></pre> Hopefully, the example helps get you up and running quickly. Flot Multi-series Bar Graph Support 2009-08-22T00:00:00+00:00 Hey boys and girls. I just got finished coding support for Multi-series bar graphs into Flot (a jQuery based JavaScript graphing library). Hence, I figured I would share not only the code but a little screenshot of it below. The code for it can be found here</a>. Below is an example of how to use it. With the multi-series bar graph support, you can have Flot display the series side by side. This is useful when you are interested in visually comparing series and grouping segments you want to see columns in series that you are specifically interested in comparing. The above screenshot was produced by defining a few code snippets. The first snippet is a snippet of HTML that includes the customized Flot javascript source file. Note: The Flot javascript source file needs to come after the standard jQuery javascript source file. An example of this snippet is as follows:  <script language="javascript" type="text/javascript" src="../jquery.js"></script> <script language="javascript" type="text/javascript" src="../jquery.flot.js"></script></code></pre> Note: In the above snippet the href values are likely to be different in your case. The second code snippet is simply a single line of HTML that is the div that is to be the Multi-series Bar Graph. It is key that this div has an id that is easily referenced and a set height and width. An example of this snippet is as follows: <div id="placeholder" style="width: 375px; height: 200px;"></div></code></pre> The third code snippet needed is the javascript that defines the data, creates the Multi-series Bar Graph in the specified div, and handles the hover functionality. An example of this is seen below. <script id="source"> $(function () { var ms_data = [{"label":"FOO","data":[[0,80],[1,70],[2,100],[3,60],[4,102]]}, {"label":"BAR","data":[[0,10],[1,20],[2,30],[3,40],[4,80]]}, {"label":"CAR","data":[[0,5],[1,10],[2,15],[3,20],[4,25]]}] var ms_ticks = [[0,"1/28"],[1,"1/29"],[2,"1/30"],[3,"1/31"],[4,"1/32"]]; function plotWithOptions() { $.plot($("#placeholder"), ms_data, { bars: { show: true, barWidth: 0.6, series_spread: true, align: "center" }, xaxis: { ticks: ms_ticks, autoscaleMargin: .10 }, grid: { hoverable: true, clickable: true } }); } function showTooltip(x, y, contents) { $('').css( { position: 'absolute', display: 'none', top: y + 5, left: x + 5, border: '1px solid #fdd', padding: '2px', 'background-color': '#fee', opacity: 0.80 }).appendTo("body").show(); } plotWithOptions(); $("#placeholder").bind("plothover", function (event, pos, item) { $("#x").text(pos.x.toFixed(2)); $("#y").text(pos.y.toFixed(2)); if (item) { if (previousPoint != item.datapoint) { previousPoint = item.datapoint; $("#tooltip").remove(); var x = item.datapoint[0].toFixed(2), y = item.datapoint[1].toFixed(2); showTooltip(item.pageX, item.pageY, item.series.label + " Group id: " + Math.floor(x) + ", y = " + y + ", seriesIndex: " + item.seriesIndex); } } else { $("#tooltip").remove(); previousPoint = null; } }); $("#placeholder").bind("plotclick", function (event, pos, item) { if (item) { $("#clickdata").text("You clicked bar " + item.dataIndex + " in " + item.series.label + "."); } }); }); </script></code></pre> Hopefully the code example helps get you up and running. TextMate Autotools Screencast 2007-09-05T00:00:00+00:00 I just threw together a screencast that covers a little TextMate Bundle I have been working. The TextMate Autotools Bundle is an extension for the TextMate editor which integrates TextMate with the standard GNU Autotools suite with a specific emphasis on cross-platform building. Check it out below to see the features and how to use them in this alpha version. Note: The video for this has been lost to the inter-web somewhere.

Drew De Ponte - A backstage perspective on running a company, building & selling apps | drewdeponte.com

The Git Patch Stack Methodology

Best Practices</h2> So what should we actually be aiming for?</p>

The Conflict</h2> When you try to apply these best practices while relying primarily on feature branches, tensions quickly emerge.</p>

Hosting my Git Repositories

The Setup</h2> This portion assumes you already have an Arch Linux system setup for your server.</p> Get SSH Daemeon Setup</h3> First thing we need to do on that server is installed the openssh</code> package so that we will have the ssh daemon installed.</p>

Get SSH Daemeon Setup</h3> First thing we need to do on that server is installed the openssh</code> package so that we will have the ssh daemon installed.</p>

Shower Thoughts 2026-01-01

Install multiple PostgreSQL versions from src on Arch

Fix PostgreSQL Collation version miss-match

Install Ruby on Arch Linux

Upgrade PostgreSQL on Arch Linux

Pull Request Alternative Design Doc

Git Commit Creation

Anatomy of a Commit</h3> To start we really need to understand what commits are in Git. Technically from a Git perspective a commit is a snapshot of a project's contents, files & folders, at a moment in time combined with a message.</p>

Snapshots & Diffs</h4> In the Git repository these snapshots are related to one another through the concept of ancestry. Meaning that generally each commit has one or more parent commits. These parent commits are the ancestors of the commit.</p>

When to rewrite Git history?

Reasonable Code Organization

My Keyboard

TLDR</h2> The following is a deep dive into my current keyboard which is outlined below.</p>

The Build</h2> The following are a bunch of photos I took throughout the build. I won't go into all the details about them. But figured I would share.</p> </p>

Notes</h2> The following are really more notes for me about things. But it might be useful to you if you are trying to get a Dactyl Manuform setup with QMK Firmware.</p>

Build Videos</h3>

QMK Firmware</h3>

Flash</h4> Connect the keyboard directly to the laptop via straight USB-C cable with the left half plugged into the computer. The right half can stay connected to the left half via the telephone wire.</li>

Edit</h4> Edit the following file to make changes to the layout.</p>

Past Keyboards</h2> I have had many past keyboards. The following is a list of just a few.</p> Kinesis Advantage2 Signature Series</a></li> ErgoDox</a></li> Infinity ErgoDox</a></li> Moonlander</a></li> Planck</a></li> Glove80</a></li> </ul>

Git Patch Stack 7.0.0

Integrate Verify Hook</h3> As part of 7.0.0</code> we now have the integrate_verify</code> hook. This is a hook that gets executed as part of the gps integrate</code> commands lifecycle. Enabling you to verify external state, maybe on an SCM hosting provider, is correct prior to integrating.</p>

Changed</h4> unhandled error output for the list command</li> unhandled error printing to print the chain of errors</li> gps --help docs to describe core vs. ancillary cmds</li>

Building a StoryBrand: Chapter 4

Actions</h2>

Application</h2>

Getting Started with StoryBranding

Pullwalla - More Workflows

Git Patch Stack Goals

Git Workflows

Cursor based Pagination with multi-key cursor

Cursor based Pagination</h2> Because of our use case we can use a strategy called Cursor Based pagination. In this strategy we pass a "cursor" identifying the transaction of the last item in the previous pages results and a limit to get back the next page.</p>

Inversion of Control

Best Practices</h2>
So what should we actually be aiming for?</p>

The Conflict</h2>
When you try to apply these best practices while relying primarily on feature branches, tensions quickly emerge.</p>

Anatomy of a Commit</h3>
To start we really need to understand what commits are in Git. Technically from a Git perspective a commit is a snapshot of a project's contents, files & folders, at a moment in time combined with a message.</p>

The Build</h2>
The following are a bunch of photos I took throughout the build. I won't go into all the details about them. But figured I would share.</p>
</p>

Notes</h2>
The following are really more notes for me about things. But it might be useful to you if you are trying to get a Dactyl Manuform setup with QMK Firmware.</p>

`Past Keyboards</h2> I have had many past keyboards. The following is a list of just a few.</p> Kinesis Advantage2 Signature Series</a></li> ErgoDox</a></li> Infinity ErgoDox</a></li> Moonlander</a></li> Planck</a></li> Glove80</a></li> </ul>`

Cursor based Pagination</h2>
Because of our use case we can use a strategy called Cursor Based pagination. In this strategy we pass a "cursor" identifying the transaction of the last item in the previous pages results and a limit to get back the next page.</p>