Easier work with Git submodules with in-place push access – Git Tricks #2

Get know how to solve issue with pushing to submodules directly from the main repo while keeping the project easily cloneable by external contributors.

Introduction

The Git submodules mechanism is pretty handy to keep the source code of lousily related dependent software together in one Git repository while leaving their development separate. It is something like symlinks in the Unix world, but with an ability to refer also to the previous version. It’s quite popular in the projects using source code integration (instead of shared libraries) or to speed up development by making related changes in multiple repositories easier. It is not the only possible solution – Gradle, for instance, provides a composite build mechanism. Monorepo is an another approach, but it has its own limitations and it very problematic to use in FOSS projects developed by different people/teams.

As usual, I encountered that situation in one of my projects. As a big enthusiast of automatic code testing and Continuous Delivery, some time ago I have been working on improving the reliability of my (automatically released) gradle-nexus-staging-plugin. After each commit (so also before the release) I wanted to have the end-to-end tests executed to verify that the plugin is able to pass a simple (but real) project through the release process to Maven Central (aka The Central Repository).

I could move that test project to the plugin repository, but – well – it’s a distinct project which could be also released separately or replaced with some other project. In addition, testing two variants of releasing it was handy to use keep it in two branches “mounted” twice in my root repository.

Problem (aka Challenge)

gradle-nexus-staging-plugin contains a submodule nexus-at-minimal from a separate repo. Working on new fancy feature in GNSP it is needed to tweak the acceptance testing project. To make the development smoother it’s very useful to be able to commit introduces changes in the dependent project directly from the working copy of the main project. It works out of the box. However, later on we should also directly push them back to a separate repo of that dependent project (by stepping into the submodule and calling git push or – all-in-once by – using the git push --recurse-submodules=on-deamand parameter when pushing from the main repository). And then – in the long term – we encounter some inconvenience.

Let’s start by defining a submodule path to connect via SSH:

In general it works fine and pushing is allowed. However, any non-developer trying to clone that repo (and initialize submodules) gets:

It’s pretty bad in the open source (FOSS) development where the projects are publicly available other people are encouraged to download (clone) it, build and contribute.

The obvious remedy to the previous error is switching to HTTPS:

However then, to allow developers to commit changes directly to a submodule it is required to go through a separate HTTPS authentication which in the most cases is completely not use in favor of SSH (and would need to be configured independently with an access token).

Transparent solution

To keep external contributors happy the developers could manually change the url in .gitmodules from HTTPS to SSH for every cloned project. However, it’s quite tedious. The better solution is an usage of pushInsteadOf. For the aforementioned example, the developers need only to add to the global ~/.gitconfig configuration file:

It effectively overrides the push URL to use SSH instead of HTTPS for the whole group in GitLab/GitHub, covering also submodules (where HTTPS – external contributors friendly scheme – is left as a default).

Summary

URL rewriting and conditional configuration (covered in the first part) are just a subset of options available in Git to make the development more flexible and simple. Simple, assuming we already found the required features and learned how to use it ;-).

The lead photo based on the Mohamed Hassan‘s work published in Pixabay, Pixabay License.

Do you want to know how to develop your skillset to become a Java Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

1. JPA Mini Book

2. JVM Troubleshooting Guide

3. JUnit Tutorial for Unit Testing

4. Java Annotations Tutorial

5. Java Interview Questions

6. Spring Interview Questions

7. Android UI Design

and many more ....

I agree to the Terms and Privacy Policy

Sign up

Thank you!

We will contact you soon.