Software Development

Easier work with Git submodules with in-place push access – Git Tricks #2

Get know how to solve issue with pushing to submodules directly from the main repo while keeping the project easily cloneable by external contributors.

Introduction

The Git submodules mechanism is pretty handy to keep the source code of lousily related dependent software together in one Git repository while leaving their development separate. It is something like symlinks in the Unix world, but with an ability to refer also to the previous version. It’s quite popular in the projects using source code integration (instead of shared libraries) or to speed up development by making related changes in multiple repositories easier. It is not the only possible solution – Gradle, for instance, provides a composite build mechanism. Monorepo is an another approach, but it has its own limitations and it very problematic to use in FOSS projects developed by different people/teams.

As usual, I encountered that situation in one of my projects. As a big enthusiast of automatic code testing and Continuous Delivery, some time ago I have been working on improving the reliability of my (automatically released) gradle-nexus-staging-plugin. After each commit (so also before the release) I wanted to have the end-to-end tests executed to verify that the plugin is able to pass a simple (but real) project through the release process to Maven Central (aka The Central Repository).

I could move that test project to the plugin repository, but – well – it’s a distinct project which could be also released separately or replaced with some other project. In addition, testing two variants of releasing it was handy to use keep it in two branches “mounted” twice in my root repository.

01
02
03
04
05
06
07
08
09
10
11
12
13
gradle-nexus-staging-plugin
├── src
│   ├── funcTest
│   │   ├── groovy
│   │   │   └── ...
│   │   └── resources
│   │       └── sampleProjects
│   │           ├── nexus-at-minimal (submodule - master branch)
│   │           │   └── ...
│   │           ├── nexus-at-minimal-publish (submodule - publish branch)
│   │           │   └── ...
│   │           └── ...
│   ├── ...

Problem (aka Challenge)

gradle-nexus-staging-plugin contains a submodule nexus-at-minimal from a separate repo. Working on new fancy feature in GNSP it is needed to tweak the acceptance testing project. To make the development smoother it’s very useful to be able to commit introduces changes in the dependent project directly from the working copy of the main project. It works out of the box. However, later on we should also directly push them back to a separate repo of that dependent project (by stepping into the submodule and calling git push or – all-in-once by – using the git push --recurse-submodules=on-deamand parameter when pushing from the main repository). And then – in the long term – we encounter some inconvenience.

Let’s start by defining a submodule path to connect via SSH:

1
2
3
[submodule "src/funcTest/resources/sampleProjects/nexus-at-minimal"]
        path = src/funcTest/resources/sampleProjects/nexus-at-minimal
        url = git@gitlab.com:nexus-at/nexus-at-minimal.git

In general it works fine and pushing is allowed. However, any non-developer trying to clone that repo (and initialize submodules) gets:

1
2
3
4
5
6
...
git@gitlab.com: Permission denied (publickey).
fatal: Could not read from remote repository.
 
Please make sure you have the correct access rights
and the repository exists.

It’s pretty bad in the open source (FOSS) development where the projects are publicly available other people are encouraged to download (clone) it, build and contribute.

The obvious remedy to the previous error is switching to HTTPS:

1
2
3
[submodule "src/funcTest/resources/sampleProjects/nexus-at-minimal"]
        path = src/funcTest/resources/sampleProjects/nexus-at-minimal
        url = https://gitlab.com/nexus-at/nexus-at-minimal.git

However then, to allow developers to commit changes directly to a submodule it is required to go through a separate HTTPS authentication which in the most cases is completely not use in favor of SSH (and would need to be configured independently with an access token).

Transparent solution

To keep external contributors happy the developers could manually change the url in .gitmodules from HTTPS to SSH for every cloned project. However, it’s quite tedious. The better solution is an usage of pushInsteadOf. For the aforementioned example, the developers need only to add to the global ~/.gitconfig configuration file:

1
2
[url "git@gitlab.com:nexus-at/"]
    pushInsteadOf = https://gitlab.com/nexus-at/

It effectively overrides the push URL to use SSH instead of HTTPS for the whole group in GitLab/GitHub, covering also submodules (where HTTPS – external contributors friendly scheme – is left as a default).

Summary

URL rewriting and conditional configuration (covered in the first part) are just a subset of options available in Git to make the development more flexible and simple. Simple, assuming we already found the required features and learned how to use it ;-).

The lead photo based on the Mohamed Hassan‘s work published in Pixabay, Pixabay License.

Published on Java Code Geeks with permission by Marcin Zajaczkowski, partner at our JCG program. See the original article here: Easier work with Git submodules with in-place push access – Git Tricks #2

Opinions expressed by Java Code Geeks contributors are their own.

Marcin Zajaczkowski

Marcin is an experienced architect who specializes in creating high quality software. Being under the impression of the Agile methodologies and the Software Craftsmanship movement, he believes in the value of good, testable and maintainable code. He aims to forge good software that makes the client delighted and the team proud of how the code itself looks.In his teaching, as a conference speaker, college lecturer, IT coach and trainer, he shows how to guide software development effectively using tests (with TDD, pair programming, Clean Code, design patterns, etc.) and maintaining a quality-oriented development environment (with CI, Sonar, automatic deployment, etc.).He is also a FOSS projects author and contributor, a Linux enthusiast.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button