Skip to content

Update nixpkgs / simplify static build #3865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
strategy:
fail-fast: false
matrix:
pgVersion: [12, 13, 14, 15, 16, 17]
pgVersion: [13, 14, 15, 16, 17]
name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04
defaults:
Expand Down
2 changes: 0 additions & 2 deletions default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ let
allOverlays.build-toolbox
allOverlays.checked-shell-script
allOverlays.gitignore
allOverlays.postgresql-libpq
(allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat
];
Expand All @@ -51,7 +50,6 @@ let
{ name = "postgresql-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-12"; postgresql = pkgs.postgresql_12.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
];

# Dynamic derivation for PostgREST
Expand Down
2 changes: 1 addition & 1 deletion docs/how-tos/sql-user-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
SQL User Management
===================

As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. Its also possible to support logins entirely through SQL. Its a fair bit of work, so get ready.
As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. It's also possible to support logins entirely through SQL. It's a fair bit of work, so get ready.

Storing Users and Passwords
---------------------------
Expand Down
2 changes: 1 addition & 1 deletion docs/references/api/domain_representations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -173,4 +173,4 @@ Domain Representations avoid all the above drawbacks. Their only drawback is tha

Why not create a `base type <https://www.postgresql.org/docs/current/sql-createtype.html#id-1.9.3.94.5.8>`_ instead? ``CREATE TYPE app_uuid (INTERNALLENGTH = 22, INPUT = app_uuid_parser, OUTPUT = app_uuid_formatter)``.

Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets how the data is presented dictate how the data is stored which would be backwards.
Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets "how the data is presented" dictate "how the data is stored" which would be backwards.
Copy link
Member

@steve-chavez steve-chavez Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange how these right/left double quotation marks were present, I don't remember using them. Perhaps we should restrict these and also for docs? For consistency and CI?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they would break CI now, yes. With an odd error, but they would do so. Not sure whether need to add something to get better feedback, I think we will remember once we hit that again.

2 changes: 1 addition & 1 deletion docs/references/api/resource_embedding.rst
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Since the table name is plural, we can be more accurate by making it singular wi
One-to-many relationships
-------------------------

The **foreign key reference** establishes the inverse one-to-many relationship. In this case, ``films`` returns as a JSON array because of the to-many end.
The **foreign key reference** establishes the inverse one-to-many relationship. In this case, ``films`` returns as a JSON array because of the "to-many" end.

.. code-block:: bash

Expand Down
2 changes: 1 addition & 1 deletion docs/references/auth.rst
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Symmetric Keys
~~~~~~~~~~~~~~

Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase, which can be configured with :ref:`jwt-secret`.
If it is set to a simple string value like reallyreallyreallyreallyverysafe then PostgREST interprets it as an HMAC-SHA256 passphrase.
If it is set to a simple string value like "reallyreallyreallyreallyverysafe" then PostgREST interprets it as an HMAC-SHA256 passphrase.

.. _asym_keys:

Expand Down
2 changes: 1 addition & 1 deletion docs/references/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,7 @@ db-pool-max-idletime
**In-Database** `n/a`
=============== =================================

*For backwards compatibility, this config parameter is also available as db-pool-timeout.*
*For backwards compatibility, this config parameter is also available as "db-pool-timeout".*

Time in seconds to close idle pool connections.

Expand Down
4 changes: 2 additions & 2 deletions docs/references/connection_pool.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Dynamic Connection Pool

To conserve system resources, PostgREST uses a dynamic connection pool. This enables the number of connections in the pool to increase and decrease depending on request traffic.

- If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that its pointless to set this higher than the ``max_connections`` setting in your database.
- If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that it's pointless to set this higher than the ``max_connections`` setting in your database.
- If a connection is unused for a period of time (:ref:`db-pool-max-idletime`), it will be released.
- For connecting to the database, the :ref:`authenticator <roles>` role is used. You can configure this using :ref:`db-uri`.

Expand Down Expand Up @@ -106,4 +106,4 @@ Also set :ref:`db-channel-enabled` to ``false`` since ``LISTEN`` is not compatib

.. note::

Its not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool.
It's not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool.
2 changes: 1 addition & 1 deletion docs/references/listener.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Listener
########

PostgREST uses `LISTEN <https://www.postgresql.org/docs/current/sql-listen.html>`_ to reload its :ref:`Schema Cache <schema_reloading_notify>` and :ref:`Configuration <config_reloading_notify>` via `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
This is useful in environments where you cant send SIGUSR1 or SIGUSR2 Unix Signals.
This is useful in environments where you can't send SIGUSR1 or SIGUSR2 Unix Signals.
Like on cloud managed containers or on Windows systems.

.. code:: postgresql
Expand Down
24 changes: 12 additions & 12 deletions nix/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,12 @@ The PostgREST utilities available in `nix-shell` all have names that begin with
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-12
postgrest-lint postgrest-with-postgresql-13
postgrest-run postgrest-with-postgresql-14
postgrest-style postgrest-with-postgresql-15
postgrest-style-check postgrest-with-postgresql-16
postgrest-test-io postgrest-with-postgresql-17
postgrest-coverage postgrest-with-postgresql-13
postgrest-lint postgrest-with-postgresql-14
postgrest-run postgrest-with-postgresql-15
postgrest-style postgrest-with-postgresql-16
postgrest-style-check postgrest-with-postgresql-17
postgrest-test-io
...

[nix-shell]$
Expand All @@ -99,12 +99,12 @@ $ nix-shell --arg memory true
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-12
postgrest-lint postgrest-with-postgresql-13
postgrest-run postgrest-with-postgresql-14
postgrest-style postgrest-with-postgresql-15
postgrest-style-check postgrest-with-postgresql-16
postgrest-test-io postgrest-with-postgresql-17
postgrest-coverage postgrest-with-postgresql-13
postgrest-lint postgrest-with-postgresql-14
postgrest-run postgrest-with-postgresql-15
postgrest-style postgrest-with-postgresql-16
postgrest-style-check postgrest-with-postgresql-17
postgrest-test-io
postgrest-test-memory
...

Expand Down
61 changes: 0 additions & 61 deletions nix/libpq.nix

This file was deleted.

8 changes: 4 additions & 4 deletions nix/nixpkgs-version.nix
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
{
owner = "NixOS";
repo = "nixpkgs";
ref = "refs/heads/nixpkgs-unstable-darwin";
date = "2024-11-09";
rev = "a90280100f41a10914edfe729a4053e60c92b8e3";
tarballHash = "1vwr665b6l6gma24w45q5hic86vbd8alc01mziwwr621hwlca88f";
ref = "refs/heads/nixpkgs-unstable";
date = "2025-01-11";
rev = "32af3611f6f05655ca166a0b1f47b57c762b5192";
tarballHash = "0shknvd56nfqh4awklgsxwaavpfixgh766m428qdlxihjmmqvhbl";
}
5 changes: 3 additions & 2 deletions nix/overlays/checked-shell-script/checked-shell-script.nix
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
, coreutils
, git
, lib
, moreutils
, runCommand
, shellcheck
, stdenv
Expand Down Expand Up @@ -56,7 +57,7 @@ let
# Example: This way `postgrest-watch -h` will return the help output for watch, while
# `postgrest-watch postgrest-test-spec -h` will return the help output for test-spec.
# Taken from: https://github.com/matejak/argbash/issues/114#issuecomment-557108274
sed '/_positionals_count + 1/a\\t\t\t\tset -- "''${@:1:1}" "--" "''${@:2}"' -i $out
sed '/_positionals_count + 1/a\\t\t\t\tset -- "''${@:1:1}" "--" "''${@:2}"' $out | ${moreutils}/bin/sponge $out
'';

bash-completion =
Expand All @@ -66,7 +67,7 @@ let
''

+ lib.optionalString (positionalCompletion != "") ''
sed 's#COMPREPLY.*compgen -o bashdefault .*$#${escape positionalCompletion}#' -i $out
sed 's#COMPREPLY.*compgen -o bashdefault .*$#${escape positionalCompletion}#' $out | ${moreutils}/bin/sponge $out
''
);

Expand Down
1 change: 0 additions & 1 deletion nix/overlays/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@
checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix;
postgresql-libpq = import ./postgresql-libpq.nix;
slocat = import ./slocat.nix;
}
21 changes: 13 additions & 8 deletions nix/overlays/haskell-packages.nix
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,20 @@ let

jose-jwt = prev.jose-jwt_0_10_0;

postgresql-libpq = lib.dontCheck (prev.callHackageDirect
postgresql-libpq = lib.overrideCabal
(lib.dontCheck (prev.callHackageDirect
{
pkg = "postgresql-libpq";
ver = "0.10.1.0";
sha256 = "sha256-tXOMqCO8opMilI9rx0D+njqjIjbZsH168Bzb8Aq8Ff4=";
}
{ }
))
{
pkg = "postgresql-libpq";
ver = "0.10.1.0";
sha256 = "sha256-tXOMqCO8opMilI9rx0D+njqjIjbZsH168Bzb8Aq8Ff4=";
}
{
postgresql = super.libpq;
});
configureFlags = [ "-fuse-pkg-config" ];
libraryPkgconfigDepends = [ super.postgresql_16 ];
librarySystemDepends = [ ];
};
};
in
{
Expand Down
6 changes: 0 additions & 6 deletions nix/overlays/postgresql-libpq.nix

This file was deleted.

64 changes: 11 additions & 53 deletions nix/static.nix
Original file line number Diff line number Diff line change
Expand Up @@ -8,64 +8,22 @@ let
inherit (pkgs) pkgsStatic;
inherit (pkgsStatic.haskell) lib;

packagesStatic =
pkgsStatic.haskell.packages."${compiler}".override (old: {
ghc = pkgsStatic.pkgsBuildHost.haskell.compiler."${compiler}".override {
# Using the bundled libffi generally works better for cross-compiling
libffi = null;
# Building sphinx fails on some platforms
enableDocs = false;
# Cross compiling with native bignum works better than with gmp
enableNativeBignum = true;
};

overrides = pkgs.lib.composeExtensions old.overrides (_: prev: {
postgresql-libpq = (lib.overrideCabal prev.postgresql-libpq {
# TODO: This section can be simplified when this PR has made it's way to us:
# https://github.com/NixOS/nixpkgs/pull/286370
# Additionally, we need to use the default version in nixpkgs, otherwise the
# override will not be active as well.
# Using use-pkg-config flag, because pg_config won't work when cross-compiling
configureFlags = [ "-fuse-pkg-config" ];
# postgresql doesn't build in the fully static overlay - but the default
# derivation is built with static libraries anyway.
libraryPkgconfigDepends = [ pkgsStatic.libpq ];
librarySystemDepends = [ ];
}).overrideAttrs (_: prevAttrs: {
buildInputs = prevAttrs.buildInputs ++ [ pkgsStatic.openssl ];
});
});
});
packagesStatic = pkgsStatic.haskell.packages.native-bignum."${compiler}";

makeExecutableStatic = drv: pkgs.lib.pipe drv [
lib.compose.justStaticExecutables

# To successfully compile a redistributable, fully static executable we need to:
# 1. make executable really statically linked.
# 2. avoid any references to /nix/store to prevent blowing up the closure size.
# 3. be able to run the executable.
# When checking for references, we ignore the following:
# - eeee... are removed references which don't actually exist
# - openssl-etc references are purposely designed to be very small
(lib.compose.overrideCabal (drv: {
postFixup = drv.postFixup + ''
exe="$out/bin/postgrest"

if ! (file "$exe" | grep 'statically linked') then
echo "not a static executable, ldd output:"
ldd "$exe"
exit 1
fi

echo "Checking for references to /nix/store..."
(${pkgsStatic.binutils}/bin/strings "$exe" \
| grep -v /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee \
| grep -v -etc/etc/ssl \
| grep /nix/store || exit 0 && exit 1)
echo "No references to /nix/store found"

"$exe" --help
'';
# 1. avoid any references to /nix/store to prevent blowing up the closure size.
# 2. be able to run the executable.
(drv: drv.overrideAttrs (finalAttrs: {
allowedReferences = [
pkgsStatic.openssl.etc
];

passthru.tests.version = pkgsStatic.testers.testVersion {
package = finalAttrs.finalPackage;
};
}))
];

Expand Down
9 changes: 9 additions & 0 deletions nix/tools/docs.nix
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ let
workingDir = "/docs";
}
''
# https://github.com/sphinx-doc/sphinx/issues/11739
export LC_ALL=C

function build() {
${python}/bin/sphinx-build --color -W -a -n . -b "$@"
}
Expand Down Expand Up @@ -119,6 +122,8 @@ let
workingDir = "/docs";
}
''
export LC_ALL=C

FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')

# shellcheck disable=SC2086 disable=SC2016
Expand All @@ -139,6 +144,8 @@ let
workingDir = "/docs";
}
''
export LC_ALL=C

FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')

tail -n+2 postgrest.dict \
Expand All @@ -157,6 +164,8 @@ let
workingDir = "/docs";
}
''
export LC_ALL=C

${python}/bin/sphinx-build --color -b linkcheck . ../.docs-build
'';

Expand Down
Loading
Loading