pkgsrc Design Goals

### [@schmonz](https://twitter.com/schmonz)
]

---

### [@schmonz](https://twitter.com/schmonz)
]

---

# pkgsrc Design Goals

## Amitai Schlair, The NetBSD Project

### pkgsrcCon London 2014/06/21

---

### Future of pkgsrc (1/2)

#### Probably not dead

* Far too useful

#### Packages

* Updated
* New ones

#### Platforms

* Better support
* New ones
]

---

### Future of pkgsrc (2/2)

#### All of the above

#### Plus, easy to:

* Discover
* Try using
* Do _all_ the basics
* Discover neat tricks
* Try developing the basics
* Discover neat bits of infrastructure
* Develop the infrastructure
]

---

### Getting to "easy" won't be easy

[Kent Beck](https://twitter.com/KentBeck/status/250733358307500032):

> for each desired change, make the change easy (warning: this may
be hard), then make the easy change

### But this is familiar territory

* We've done hard things in order to reach the payoff
* Abstractions
* Buildlink
* Wrappers
* Now cross-platform support is relatively easy

]

---

### Credit where due

These ideas mostly came from recent discussions with `gavan`,
`jaapb`, and `agc`. I'm sure others have helped us arrive at these
ideas, I'm just not sure exactly who or when.

These aren't the only ideas. They might not even be good ones.
That's why:

7. We ought to try them
7. You should feel comfortable sharing your ideas too

]

---

### Don't rewrite the whole infrastructure

* Far too much work
* Impossible to undertake
* We want to be sure to keep the functionality we have

]

---

### _Do_ rewrite the infrastructure

* But not all at once

]

---

### Rewrite the infrastructure, piece by piece

* Try to, anyway
* Discover that the interface between your piece and the rest is
  "spaghetti with shared-global-state sauce"
* Define a healthier interface with automated tests
* Refactor till your tests pass

### Achieve greater clarity

* The code may not have magically become clear
* The interface sure did
* The tests sure help

]

---

### Build up a suite of tests for infrastructure

* Side effect of bulk builds: end-to-end test results
* Missing effects of bulk builds:
    * How can I comfortably experiment with infrastructure code?
    * How can I understand what's expected from this code?
    * How can I quickly find out if I've broken it?
    * How can I decide whether my changes might be good to commit?
* Hence a suite of smaller, faster, more specific tests
    * That we've developed, piece by piece, as we needed them

]

---

### Try alternate implementations

* Given defined interfaces and fine-grained tests...
* Possible to try other ways to satisfy an interface
* Say, using an implementation language other than `make`
* `make` hasn't been a wonderful implementation language for us
* Perhaps `make` isn't the best UI, either, but we could keep it

]

---

### Discover a better implementation language

* Must be as portable as pkgsrc itself
* Must be easily bootstrapped
* But we don't have to derive the best choice from first principles
    * And we're wise not to try
* We can observe it experimentally!
* Look at working alternate implementations of infrastructure bits
* Discover our real constraints and how constraining they truly are
* Move, bit by bit, to code we like living with

]

---

### Migrate to declarative `Makefile`

* An imperative part
* A declarative part
* Easy to reason about the declarative bits
    * e.g., calculate dependencies early and fast
    * difficult to create package-specific bugs
* Someday, no imperative part
    * `pkglint` can help encourage this

### Migrate to declarative `buildlink.mk`

### Manage the transition

* Perhaps generate input to `make` until all migrated

]

---

### Update packages safely and easily

* Smash together `pkg_rolling-replace(8)`, `pkg_comp(8)`, pkgviews,
  `DESTDIR`, wrappers
* Avoid needing root, some kind of jail
* Avoid always needing worst-case disk space
* Avoid failing partway through, leaving inconsistent state
* Minimize before-to-after downtime
* Assume most dependencies can be run from any installed path (or
  don't need to be)
* Say `make update-everything-or-nothing`
* Build needed dependencies (where they don't match installed) as
  binary packages
* Install needed dependencies to a temporary path under `${WRKDIR}`
* Point buildlink and the wrappers at that path
* Done? Replace binary packages with new ones. Or else don't.

]

---

### Cross-build packages more easily

* Much (undersung) work already done
* Install the right cross-compiler in the temporary path

### Bootstrap more easily

* For impoverished platforms...
* Build a minimal `gcc`
* Use it to build a minimal `gmake`
* Use it to build a better `gcc`
* And so on, until a full `gcc` and `gmake` are installed
* Or for any platform...
* `ghc` needs a working `ghc` to build

### Resolve dependencies more reliably

* By reasoning from declarative dependencies
* By inspecting available binary packages

]

---

### The future I want

#### We've made it hard to develop the features we want and need

Inertia is currently working against us.

#### We can make things easier, one piece at a time

#### We can develop the features we want and need

Inertia will start working _for_ us.

#### We'll be easy to use, develop, and promote

]

---

# FIN