Even large companies are still grasping at straws when it comes to good code. Meanwhile there are articles I wrote years ago which explain clearly from first principles why the correct philosophy is "Generic core, specific shell."
I actually remember early in my career working for a small engineering/manufacturing prototyping firm which did its own software, there was a senior developer there who didn't speak very good English but he kept insisting that the "Business layer" should be on top. How right he was. I couldn't imagine how much wisdom and experience was packed in such simple, malformed sentences. Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
Your advice is the opposite of "functional core, imperative shell". The FCIS principle has IS which is generic, to be simple, because it's usually hard to test (it deals with resources and external dependencies). So by being simple, it's more unit testable.
On the other hand, FC is where the business logic lives, which can be complex and specific. The reason why you want that "functional" (really just another name for "composable from small blocks") is because it can be tested for validity without external dependencies.
So the IS shields you from technicalities of external dependencies, like what kind of quirks your DB has, or are we sending data over network or writing to the file, or does the user inputs comands in spanish or english, do you display the green square or blue triangle to indicate the report is ready, etc.
On the other hand, FC deals with the actual business logic (what you want to do), which can be both generic and specific. These are just different types of building blocks (we call them functions) living in the FC.
FCIS is exemplified by user-shell interaction. The user (FC) dictates the commands and interprets the output, according to her "business needs". While the shell (IS) simply runs the commands, without any questions of their purpose. It's not the job of IS to verify or handle user errors caused by wrong commands.
But the user doesn't do stuff on her own; you could take her to a pub and she would tell you the same sequence of commands when facing the same situation. In that sense, the user is "functional" - independent on the actual state of the computer system, like the return value of a mathematical function is only dependent on the arguments.
Another example is MVC, where M is the FC and VC is the IS. Although it's not always exactly like that, for variety of reasons.
You can think of IS as a translator to a different language, understood by "the other systems", while the FC is there to supply what is actually being communicated.
I disagree that these two pieces of advice are opposed. I think they are orthogonal at worst, and in agreement at best.
"Functional core, imperative shell" (FCIS) is a matter of implementing individual software components that need to engage with side-effects --- that is, they have some impact on some external resources. Rather than threading representations of the external resources throughout the implementation, FCIS tells us to expel those concerns to the boundary. This makes the bulk of the component easier to reason about, being concerned with pure values and mere descriptions of effects, and minimizes the amount of code that must deal with actual effects (i.e. turning descriptions of effects into actual effects). It's a matter of comprehensibility and testability, which I'll clumsily categorize as "verification": "Does it do what it's supposed to do?"
"Generic core, specific shell" (GCSS) is a matter of addressing needs in context. The problems we need solved will shift over time; rather than throwing away a solution and re-solving the new problem from scratch, we'd prefer to only change the parts that need changing. GCSS tells us we shouldn't simply solve the one and only problem in front of us; we should use our eyes and ears and human brains to understand the context in which that problem exists. We should produce a generic core that can be applied to a family of related problems, and adapt that to our specific problem at any specific time using a, yes, specific shell. It's a matter of adaptability and solving the right problem, which I'll clumsily categorize as "validation": "Is what it's supposed to do what we actually need it to do?"
Ideally, GCSS is applied recursively: a specific shell may adapt an only slightly more generic core, which then decomposes into a smaller handful of problems that are themselves implemented with GCSS. When business needs change in a way that the outermost "generic core" can't cover, odds are still good that some (or all) of its components can still be applied in solving the new top-level problem. FCIS isn't really amenable to the same recursion.
Both verification and validation activities are necessary. One is a matter of internal consistency within the component; the other is a matter of external consistency relative to the context the component is being used in. FCIS and GCSS advise on how to address each concern in turn.
> I think they are orthogonal at worst, and in agreement at best.
I have considered them being orthogonal, but then the definition of the "shell" and "core" becomes problematic in this comparison. What you call shell in GCSS is not shell in FCIS at all, more like a boundary. Even there it is questionable whether boundary should be more specific than the core. At the core, things can be more integrated than at the boundary, and so it can have more business-specific rules.
I am not disputing GP's advice, but I feel it is perhaps a little bit simplistic if not tautological ("prefer generic building blocks where possible"), and really muddles up what the core and shell is in the FCIS meaning.
I've seen it many times. And then every task takes longer than the last one, which is what pushes teams to start rewrites.
"There's never enough time to do it right, but always time to do it again."
> Probably many reasons for this, but what I've seen often is that once the code base has been degraded, it's a slippery slope downhill after that.
Another factor, and perhaps the key factor, is that contrary to OP's extraordinary claim there is no such thing as objectively good code, or one single and true way of writing good code.
The crispest definition of "good code" is that it's not obviously bad code from a specific point of view. But points of view are also subjective.
Take for example domain-driven design. There are a myriad of books claiming it's an effective way to generate "good code". However, DDD has a strong object-oriented core, to the extent it's nearly a purist OO approach. But here we are, seeing claims that the core must be functional.
If OP's strong opinion on "good code" is so clear and obvious, why are there such critical disagreements at such a fundamental levels? Is everyone in the world wrong, and OP is the poor martyr that is cursed with being the only soul in the whole world who even knows what "good code" is?
Let's face it: the reason there is no such thing as "good code" is that opinionated people making claims such as OP's are actually passing off "good code" claims as proxy's for their own subjective and unverified personal taste. In a room full of developers, if you throw a rock at a random direction you're bound to hit one or two of these messiahs, and neither of them agrees on what good code is.
Hearing people like OP comment on "good code" is like hearing people comment on how their regional cuisine is the true definition of "good food".
These are great and succinct, yours and your teammate’s.
I still find myself debating this internally, but one objective metric is how smoothly my longer PTOs go:
The only times I haven’t received a single emergency call were when I left teammates a a large and extremely specific set of shell scripts and/or executables that do exactly one thing. No configs, no args/opts (or ridiculously minimal), each named something like run-config-a-for-client-x-with-dataset-3.ps1 that took care of everything for one task I knew they’d need. Just double click this file when you get the new dataset, or clone/rename it and tweak line #8 if you need to run it for a new client, that kind of thing.
Looking inside the scripts/programs looks like the opposite of all of the DRY or any similar principles I’ve been taught (save for KISS and others similarly simplistic)
But the result speaks for itself. The further I go down that excessively basic path, the more people can get work done without me online, and I get to enjoy PTO. Anytime i make a slick flexible utility with pretty code and docs, I get the “any chance you could hop on?” text. Put the slick stuff in the core libraries and keep the executables dumb
I see a similar problem in infra-land where people expose too many config variables for too many things, creating more cruft. Knowing what to hardcode and what to expose as a var is something a lot of devs don't seem to understand; and don't realise they don't understand.
Yes I feel that when to apply certain techniques is frequently under-discussed. But I can't blame people for err-ing on the side of 'do everything properly' - as this makes life more pleasant in teams.
Although I think if you squint, the principle still applies to your example. The further you get from the 'core' of your platform/application/business/what-have-you, the less abstract you need to be.
> The more specific, the more brittle. The more general, the more stable. Concerns evolve/decay at different speeds, so do not couple across shearing layers. Notice how grammar/phonology (structure) changes slowly while vocabulary (functions, services) changes faster.
...
> Coupling across layers invites trouble (e.g. encoding business logic with “intuitive” names reflecting transient understanding). When requirements shift (features, regulations), library maintainers introduce breaking changes or new processor architectures appear, our stable foundations, complected with faster-moving parts, still crack!
Bertrand Meyer suggested another way to consider this that ends up in a similar place.
For concerns of code complexity and verification, code that asks a question and code that acts on the answers should be separated. Asking can be done as pure code, and if done as such, only ever needs unit tests. The doing is the imperative part, and it requires much slower tests that are much more expensive to evolve with your changing requirements and system design.
The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable, and they are easy to accidentally expose to third party code through the addition of subsequent features, even if initially they are unreachable. Sun biffed this way a couple of times with Java.
But for non crosscutting concerns this advice can also be a step toward FC/IS, both in structuring the code and acclimating devs to the paradigm. Because you can start extracting pure code sections in place.
Command-Query Separation is the term for that. However, I find this statement odd:
> having functions that do things without verifying preconditions are exploitable
Why would you do this? The separation between commands and queries does not mean that executing a command must succeed. It can still fail. Put queries inside the commands (but do not return the query results, that's the job of the query itself) and branch based on the results. After executing a command which may fail, you can follow it with a query to see if it succeeded and, if not, why not.
I think CQRS is something different than what’s being described here. “Query” code in CQRS can still “do stuff”: call an external database, grab locks, audit trail recording etc.
What’s being described here is something lower level, that you keep as much code as you can as a side-effect-free “pure functional core”. That pattern is useful both for the “command” and “query” side of a CQRS system, and is not the same thing as CQRS
In asynchronous environments, you may not be able to repeat the same query with the same result (unless you control a cache of results, which has its own issues). If some condition is determined by the command’s implementation that subsequent code is interested in (a condition that isn’t preventing the command from succeeding), it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
I can’t decide if this really is the biggest problem with CQS. Certainly the wiki page claims it is, and it’s a reasonable argument. For some simpler cases you could dodge it by wrapping the function pairs/tuples in a lock. Database calls are a bit sketchy, because a transaction only “fixes” the problem if you ignore the elephant in the room which is reduced system parallelism by a measurable amount because even in an MVCC database transactions aren’t free. They’re just cheaper.
Caches always mess up computational models because they turn all reads into writes. Which makes things you could say with static analysis no longer true. I know a lot of tricks for making systems faster and I’ve hardly ever seen anyone apply most of them to systems after caching was introduced. It has one upside and dozens of downsides as bad or worse than this one.
> it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
You don't need the command to return anything (though it can be more efficient or convenient). It can set state indicating, "Hey, I was called but by the time I tried to do the thing the world and had changed and I couldn't. Try using a lock next time."
if (query(?)) {
command(x)
result := status(x) // ShouldHaveUsedALockError
}
The caller can still obtain a result following the command, though it does mean the caller now has to explicitly retrieve a status rather than getting it in the return value.
Where is that state stored, in an environment where the same command could be executed with the same parameters but resulting in a different status, possibly in parallel? How do you connect the particular command execution with the particular resulting status? And if you manage to do so, what is actually won over the command just returning the status?
I’d argue that the separation makes things worse here, because it creates additional hidden state.
Also, as I stated, this is not about error handling.
CQRS should really only guide you to designing separate query and command interfaces. If your processing is asynchronous then you have no choice but to have state about processing-in-flight, and your commands should return an acknowledgement of successful receipt of valid commands with a unique identifier for querying progress or results. If your processing is synchronous make your life easier by just returning the result. Purity of CQRS void-only commands is presentation fodder, not practicality.
(One might argue that all RPC is asynchronous; all such arguments eventually lead to message buses, at-least-once delivery, and the reply-queue pattern, but maybe that's also just presentation fodder.)
You may have a command sub-routine that is used by multiple higher-level commands, or even called multiple times within by a higher-level command. If the validation lives in the subroutine, that validation will be called multiple times, even when it only needs to be called once.
So you are forced to choose either efficiency or the security of colocating validation, which makes it impossible to call the sub-routine with unvalidated input.
hinkley poses this as a fault in CQS, but CQS does not require your commands to always succeed. Command-Query Separation means your queries return values, but produce no effects, and your commands produce effects, but return no values. Nothing in that requires you to have a command which always succeeds or commands which don't make use of queries (queries cannot make use of commands, though). So a better question than what I originally posed:
My "Why would you do this?" is better expanded to: Why would you use CQS in a way that makes your system less secure (or safe or whatever) when CQS doesn't actually require that?
The example in the wiki page is far more rudimentary than the ones I encountered when I was shown this concept. Trivial, in fact.
CQS will rely on composition to do any If A Then B work, rather than entangling the two. Nothing forces composition except information hiding. So if you get your interface wrong someone can skip over a query that is meant to short circuit the command. The constraint system in Eiffel I don’t think is up to providing that sort of protection on its own (and the examples I was given very much assumed not). Elixir’s might end up better, but not by a transformative degree. And it remains to be seen how legible that code will be seen as by posterity.
That's still not really answering my question for you, which was less clear than intended. To restate it:
> The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable
My understanding of your comment was that "this advice" is CQS. So you're saying that CQS commands do not verify preconditions and that this is a weakness in CQS, in particular.
Where did you get the idea that CQS commands don't verify preconditions? I've never seen anything in any discussion of it, including my (admittedly 20 years ago) study of Eiffel.
Somewhere there’s a B without the associated query. Call it what you want, at the bottom of the tree two roads diverge. Otherwise there is no Separation in your CQS.
ETA: once you get down to the mutation point you aren’t just dealing with immutable data. You’re moving things around, often plural.
I like the idea but the example doesn't make much sense.
In what application would you load all users into memory from database and then filter them with TypeScript functions? And that is the problem with the otherwise sound idea "Functional core, imperative shell". The shell penetrates the core.
Maybe some filters don't match the way database is laid out, what if you have a lot of users, how do you deal with email batching and error handing?
So you have to write the functional core with the side effect context in mind, for example using query builder or DSL that matches the database conventions. Then weave it with the intricacies of your email sender logic, maybe you want iterator over the right size batches of emails to send at once, can it send multiple batches in parallel?
I am surprised by this example, for the same reason.
Generally, performance is a top cause of abstraction leaks and the emergence of less-than-beautiful code. On an infinitely powerful machine it would be easy and advisable to program using neat abstracrions, using purely "the language of" the business. Our machines are not infinitely powerful, and that is especially evident when larger data sets are involved. That's where, to achieve useful performance, you have to increasingly speak "the language of" the machine. This is inevitable, and the big part of the programmer's skill is to be able to speak both "languages", to know when to speak which one, and produce readable code regardless.
Database programming is a prime example. There's a reason, for example, why ORMs are very messy and constitute such excellent footguns: they try to gap this bridge, but inevitably fail in important ways. And having and ORM in the example would, most likely, violate the "functional core" principle from the article.
So it looks like the author accidentally presented a very good counterexample to their own idea. I like the idea though, and I would love to know how to resolve the issue.
Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say.
I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
What makes it hard to reason about is that your code is one-dimensional, you have functions like `getExpiredUsers` and `generateExpiryEmails` which could be expressed as composition of more general functions. Here is how I would have written it in JavaScript:
const emails = db.getUsers()
.filter(user => user.isExpired(Date.now())) // Some property every user has
.map(generateExpiryEmail); // Maps a single user to a message
email.bulkSend(emails);
The idea is that you have small but general functions, methods and properties and then use higher-order functions and methods to compose them on the fly. This makes the code two-dimensional. The outer dimension (`filter` and `map`) tells the reader what is done (take all users, pick out only some, then turn each one into something else) while the outer dimension tells you how it is done. Note that there is no function `getExpiredUsers` that receives all users, instead there is a simple and more general `isExpired` method which is combined with `filter` to get the same result.
In a functional language with pipes it could be written in an arguably even more elegant design:
I guess I just never encounter code like this in the big enterprise code bases I have had to weed through.
Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Do you just do the const emails =
three different times?
In my coding world it looks a lot like doing a
SELECT * ON users WHERE isExpired < Date.now
but in some cases you just grab it all, loop through it all, and do little switches to do different things based on different isExpired.
If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Well, in that case you wouldn't want to pipe them all through generateExpiryEmail.
But perhaps you can write a more generic function like generateExpiryEmailOrWhatever that understands the user object and contains the logic for what type of email to draft. It might need to output some flag if, for a particular user, there is no need to send an email. Then you could add a filter before the final (send) step.
myCoolSubroutine = do
now <- getCurrentTime
users <- getUsers
forM users (sendEmail now)
sendEmail now user =
if user.expiry <= now
then sendExpiryEmail user
else sendNonExpiryEmail user
> Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
>
> Do you just do the const emails =
>
> three different times?
If it's just two or three cases I might actually just copy-paste the entire thing. But let's assume we have twenty or so cases. I'll use Python notation because that's what I'm most familiar with. When I write `Callable[[T, U], V]` that means `(T, U) -> V`.
Let's first process one user at a time. We can define an enumeration for all our possible categories of user. Let's call this enumeration `UserCategory`. Then we can define a "categorization function" type which maps a user to its category:
type UserCategorization = Callable[[User], UserCategory]
I can then map each user to a tuple of category and user:
categorized_users = map(categorize, db.get_users()) # type Iterable[tuple[UserCategory, User]]
Now I need a mapping from user category to processing function. I'll assume we call the processing function for side effects only and that it has no return value (`None` in Python):
type ProcessingSpec = Mapping[UserCategory, Callable[[User], None]
This mapping uses the user category to look up a function to apply to a user. We can now put it all together: map each user to a pair of the user's category and the user, then for each pair use the mapping to look up the processing function:
def process_users(how: ProcessingSpec, categorize: UserCategorization) -> None:
categorized_users = map(categorize, db.get_users())
for category, user in categorized_users:
process = how[category]
process(user)
OK, that's processing one user a time, but what if we want to process users in batches? Meaning I want to get all expired users first, and then send a message to all of them at once instead of one at a time. We can actually reuse most of our code because how how generic it is. The main difference is that instead of using `map` we want to use some sort of `group_by` function. There is `itertools.groupby` in the Python standard library, but it's not exactly what we need, so let's write our own:
def group_by[T, U](what: Iterable[T], key: Callable[[T], U]) -> Mapping[U, list[T]]:
result = defaultdict(list)
# When we try to look up a key that does not exist defaultdict will create a new
# entry with an empty list under that key
for x in what:
result[key(x)].append(x)
return x
Now we can categorize our users into batches based on their category:
batches = group_by(db.get_users(), categorize)
To process these batches we need a mapping from batch to a function which process an iterable of users instead of just a single user.
type BatchProcessingSpec = Mapping[UserCategory, Callable[[Iterable[User]], None]
Now we can put it all together:
def process_batched_users(how: BatchProcessingSpec, categorize: UserCategorization) -> None:
batches = group_by(db.get_users(), categorize)
for category, users in batches:
process = how[category]
process(users)
There are quite a lot of small building block functions, and if all I was doing was sending emails to users it would not make sense to write these small function that add indirection. However, in a large application these small functions become generic building blocks that I can use in higher-order functions to define more concrete routines. The `group_by` function can be used for many other purposes with any type. The categorization function was used for both one-at-a-time and batch processing.
I have been itching to write a functional programming book for Python. I don't mean a "here is how to do FP in Python" book, you don't need that, the documentation of the standard library is good enough. I mean a "learn how to think FP in general, and we are going to use Python because you probably already know it". Python is not a functional language, but it is good enough to teach the principles and there is value in doing things with "one hand tied behind your back". The biggest hurdle in the past to learning FP was that books normally teach FP in a functional language, so now the reader has to learn two completely new things.
That then writes the "send expiry email" commands from the aggregate, to an outbox, which a worker then picks up to send. Simple, transactional domain logic.
Not a single person in this thread commented on the use of Date.now() and similar - surely clock.now() - you never ever want to use global time in any code, how could you test it?
clock in this case is a thing that was supplied to the class or function. It could just be a function: () -> Instant.
(Setting a global mock clock is too evil, so don't suggest that!)
I was just referring to how pipes make these kinds of chained function calls more readable. But on your point, I think using Date.now() is perfectly ok.
The nice thing with the Elixir example is that you can easily `tap()` to inspect how the data looks at any point in the pipeline. You can also easily insert steps into the pipeline, or reuse pipeline steps. And due to the way modules are usually organized, it would more realistically read like this, if we were in a BulkEmails module:
The nice thing here is that we can easily log to the console, and also filter out nil expiry emails. In production code, `generate_expiry_email/1` would likely return a Result (a tuple of `{:ok, email}` or `{:error, reason}`), so we could complicate this a bit further and collect the errors to send to a logger, or to update some flag in the db.
It just becomes so easy to incrementally add functionality here.
---
Quick syntax reference for anyone reading:
- Pipelines apply the previous result as the first argument of the next function
- The `/1` after a function name indicates the arity, since Elixir supports multiple dispatch
> I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
What you want is to use a language that has higher-kinded types and monads so that functions can have both effects (even multiple distinct kinds of effects) and return values, but the distinction between the two is clear, and when composing effectful functions you have to be explicit about how they compose. (You can still say "run these three possibly-erroring functions in a pipeline and return either the successful result or an error from whichever one failed", but you have to make a deliberate choice to).
Making a distinction between pure and effectful functions doesnt require any kind of effect system though.
Having a language where "func" defines a pure function and "proc" defines a procedure that can performed arbitrary side effects (as in any imperative language really) would still be really useful, I think
> Having a language where "func" defines a pure function and "proc" defines a procedure that can performed arbitrary side effects (as in any imperative language really) would still be really useful, I think
Rust tried that in the early days, the problem is no-one can agree on exactly what side effects make a function non-pure. You pay almost all the costs of a full effect system (and even have to add an extra language keyword) but get only some of the benefits.
I may have gotten nerd sniped here, but I believe all of these examples so far have some subtle errors. Using elixir syntax, I would think something like this covers most of the cases:
expiry_date = DateTime.now!("Etc/UTC")
query =
from u in User,
where:
u.expiry_date > ^expiry_date
and u.expiry_email_sent == false,
select: u
MyAppRepo.all(query)
|> Enum.map(u, &generate_expiry_emails(&1, expiry_date))
|> Email.bulkSend() # Returns {:ok, %User{}} or {:err, _reason}
|> Enum.filter(fn
{:ok, _} -> true
_ -> false
end)
|> Enum.map(fn {:ok, user} ->
User.changeset(user, %{expiry_email_sent: true})
|> Repo.update()
end)
Mainly a lot of these examples do the expiry filtering on the application side instead of the database side, and most would send expiry emails multiple times which may or may not be desired behavior, but definitely isn't the best behavior if you automatically rerun this job when it fails.
----
Edit: I actually see a few problems with this, too, since Email.bulkSend probably shouldn't know about which user each email is for. I always see a small impedance mismatch with this sort of pipeline, since if we sent the emails individually it would be easy to wrap it in a small function that passes the user through on failure.
If I were going to build a user contacting system like this I would probably want a separate table tracking emails sent, and I think that the email generation could be made pure, the function which actually sends email should probably update a record including a unique email_type id and a date last sent, providing an interface like: `send_email(user_query, email_id, email_template_function)`
On the same page here, read it multiple times to see if I can convince my mind, this is bit off in terms of reading the code as its being executed. There are high chances of people making mistakes over the time with such patterns. As usual there is always a trade off involved, readability is the one taking hit here.
That's pretty hardcore, like you want to restrict the runtime substitution of function calls with their result values? Even Haskell doesn't go that far.
Generally you'd distinguish which function call introduces the error with the function call stack, which would include the location of each function's call-site, so maybe the "low-effort" label is accurate. But I could see a benefit in immediately knowing which functions are "pure" and "impure" in terms of manipulating non-local state. I don't think it changes any runtime behavior whatsoever, really, unless your runtime schedules function calls on an async queue and relies on the order in code for some reason.
It has been so long since I worked on the code that had chaining functions and caused problems that I am not sure I can do justice to describing the problems.
I vaguely remember the problem was one function returned a very structured array dealing with regex matches. But there was something wrong with the regex where once in a blue moon, it returned something odd.
So, the chained functions did not error. It just did something weird.
Whenever weird problems would pop up, it was always passed to me. And when I looked at it, I said, well...
I am going to rewrite this chain into steps and debug each return. Then run through many different scenarios and that was how I figured out the regex was not quite correct.
You could write the logic in a more straight forward, but less composable way, so that all the logic resides in one pure function. This way you can also keep the code to only loop over the users once.
I would have written each statement on its own line:
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);
email.bulkSend(expiryEmails);
This is not only much easier to read, it's also easier to follow in a stack trace and it's easier to debug. IMO it's just flat out better unless you're code golfing.
I'd also combine the first two steps by creating a DB query that just gets expired users directly rather than fetching all users and filtering them in memory:
expiredUsers = db.getExpiredUsers(Date.now());
Now I'm probably mostly getting zero or a few users rather than thousands or millions.
This is actually closer to the way the first draft of this article was written. Unfortunately, some readability was lost to make it fit on a single page. 100% agree that a statement like this is harder to reason about and should be broken up into multiple statements or chained to be on multiple lines.
Yeah. I did not mention what I would do, but what you wrote is pretty much what I prefer. I guess nobody likes it these days because it is old procedural style.
There's nothing procedural about binding return values to variables, so long as you aren't mutating them. Every functional language lets you do that. That's `let ... in` in Haskell.
I agree because it reads as it will process in the direction I normally read. But I do think one of the benefits of the function approach is that the scope isn't cluttered with staging variables.
For these reasons one of the things I like to do in Swift is set up a function called ƒ that takes a single closure parameter. This is super minimal because Swift doesn't require parenthesis for the trailing closure. It allows me to do the above inline without cluttering the scope while also not increasing the amount of redirection using discrete function declarations would cause.
The above then just looks like this:
ƒ {
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);\
email.bulkSend(expiryEmails);
}
I would argue that the real key is to have a distinct core and shell, and to hold the core to a much higher standard of quality than the shell. In this article, being "functional" is just serving as a proxy for code quality.
I wrote our AI agents code with a functional core + imperative shell and I have to agree: this approach yields much faster cycle times because you can run pure unit tests and it makes testing a lot easier.
We have tens of thousands of lines of code for the platform and millions of workflow runs through them with no production errors coming from the core agent runtime which manages workflow state, variables, rehydration (suspend + resume). All of the errors and fragility are at the imperative shell (usually integrations).
Some of the examples in this thread I think get it wrong.
The effect of this is that the inner logic of `bulkSend` is completely decoupled from I/O and external logic. Now there's no need for mocking or integration tests because it is possible to use pure unit tests by simply swapping out the functions. I can easily unit test `bulkSend` because I don't need to mock anything or know about the inner behavior.
I chose this approach because writing integration tests with LLM calls would make the testing run too slowly (and costly!) so most of the interaction with the LLM is simply a function passed into our core where there's a lot of logic of parsing and moving variables and state around. You can see here that you no longer need mocks and no longer need to spy on calls because in the unit test, you can pass in whatever function you need and you can simply observe if the function was called correctly without a spy.
It is easier than most folks think to adopt -- even in imperative languages -- by simply getting comfortable working with functions at the interfaces of your core API. Wherever you have I/O or a parameter that would be obtained from I/O (database call), replace it with a function that returns the data instead. Now you can write a pure unit test by just passing in a function in the test.
I am very surprised how many of the devs on the team never write code that passes a function down.
I like the idea of separating your “business logic” (the functional core) from the glue code that interacts with the outside world (the imperative shell). It makes the core easier to test and reason about.
But also: the challenge is knowing where to draw the line. In real systems you’ll still have messy side-effects, transactions, performance constraints — so you might end up in a mixed bag anyway. The principle is solid, but the practical trade-offs matter.
The MirageOS project [0] is a great collection of functionality pure OCaml libraries that are useful outside of unikernels. I've used the DNS library with an effectful layer for various nameserver experiments [1].
I have written a small system in Elixir adhering to FCIS. Not used to the approach, I was pretty slow and sometimes it felt like jumping through hoops set by myself, lol, but I loved it, the code was very clean, testable, and refactorable. Highly recommend it as an exercise, it was surprising just how much state and IO can be pushed out.
What if a FCF (functional core function) calls another FCF which calls another FCF? Or do we do we rule out such calls?
Object Orientation is only a skin-deep thing and it boils down to functions with call stack. The functions, in turn, boil down to a sequenced list of statements with IF and GOTO here and there. All that boils boils down to machine instructions.
So, at function level, it's all a tree of calls all the way down. Not just two layers of crust and core.
Functional core usually means pure functional functions, aka the return value is know if the arguments is known, no side effects required. All the side effects is then pushed up the imperative shell.
You’ll find usually that side effect in imperative actions is usually tied to the dependencies (database, storage, ui, network connections). It can be quite easy to isolate those dependencies then.
It’s ok to have several layers of core. But usually, it’s quite easy to have the actual dependency tree with interfaces and have the implementation as leaves for each node. But the actual benefits is very easy testing and validation. Also fast feedback due to only unit tests is needed for your business logic.
Sometimes, sure - but sometimes, passing around a fat wrapper around a DB cursor is worse, and the code would be better off paginating and materializing each page of data in memory. As usual, it depends.
Another good blog post that is IMO in the same vein: https://lambdaisland.com/blog/2022-03-10-mechanism-vs-policy (“Improve your code by separating mechanism from policy”). This blends harmoniously with “functional core, imperative shell”—the "mechanism" code is the "functional core", and the "policy" code is the "imperative shell"—and also a little bit with John Ousterhout's idea in A Philosophy of Software Design of "deep modules" (in this context, don't put policy stuff, i.e. arbitrary decisions, inside the module).
Haskell practically encourages this style of programming. Any function that touches IO needs to wrap outputs with an appropriate monad. It becomes easier to push all IO out to the edges of your program and keep your core purely functional with no monads
I wish that's what people did, some codebases I've seen are messes of monad transformer stacks the likes of which you've never seen.
I mean, what if you want to do IO and have mutable data structures inside a do block? I'm afraid I'm going to have to prescribe you a monad transformer. Be careful of the side effects.
This works right up to the point where you try to make the code to support opening transactions functional. :D
Some things are flat out imperative in nature. Open/close/acquire/release all come to mind. Yes, the RAI pattern is nice. But it seems to imply the opposite? Functional shell over an imperative core. Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
Edit: I certainly think having some sort of affordance in place to indicate if you are in different sections is nice.
Right, but even in those, you typically have the more imperative operations as the lower levels, no? Especially when you have things where the life cycle of what you are starting is longer than the life cycle of the code that you use to do it?
Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
> but even in those, you typically have the more imperative operations as the lower levels
Yes, the monadic part is the functional core, and the runtime is the imperative shell.
> Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
What do you mean by Monad trick? That's precisely the kind of thing the IO monad exists for. If you need to fetch things on an API: IO. If you need to read/save things on a DB: IO. DB Transaction: IO.
I have not seen too many (any?) times where the monad trick is done in such a way that they don't combine everything in a single context wrapper and talk about the "abnormal" case where things don't complete during execution.
Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)
You would deal with this problem in the same way you would with, say, in a REST API.
If the transaction object is serializable you can just store it in a DB, for example. If it's some C++ pointer from some 3rd-party library that you can't really serialize and gotta keep open, you gotta keep it in memory and manage its lifetime explicitly, be it a REST web server, in Haskell or in a C++ app.
My idea here is that, in many domains, you will have operations that are somewhat definitionally in the imperative camp. OpenTransaction being the easy example.
Can you implement it using functional code? Yes. Just make sure you wind up with partial states. And often times you are best off explicitly not using the RAI pattern for some of these. (I have rarely seen examples where they deal with this. Creating and reconciling transactions often have to be separate pieces of code. And the reconcile code cannot, necessarily, fallback to create a transaction if they get a "not found" fault.)
> Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
That's not what functional core, imperative shell means though. It's a given that CPUs aren't functional. The advice is for people programming in languages that have expressions - ruby, in the case of the original talk. The functional paradigm mostly assumes automatic memory management.
Right, I was just using that as "at the extreme" and how it largely exists to allow you to put a functional feel on top of the imperative below it.
I'm sympathetic to the idea, as you can see it in most instruction manuals that people are likely to consume. The vast majority of which (all of them?) are imperative in nature. There is something about the "for the humans" layer being imperative. Step by step, if you will.
I don't know that it fully works, though. I do think you are well served being consistent in how you layer something. Where all code at a given layer should probably stick to the same styles. But knowing which should be the outer and which the inner? I'm not clear that we have to pick, here. Feel free to have more than two layers. :D
I saw Gary posted his blog link on twitter, and I really like his article. I really didn't expect it to surface up at this moment (2025), and it's referred from a google blog. :shrug:
I invented this pattern when I was working on a small ecommerce system (written in Scheme, yay!) in the early 2000s. It just became much easier to do all the pricing calculations, which were subject to market conditions and customer choices, if I broke it up into steps and verified each step as a side-effect-free, data-in-data-out function.
Of course by "invented" I mean that far smarter people than me probably invented it far earlier, kinda like how I "invented" intrusive linked lists in my mid-teens to manage the set of sprites for a game. The idea came from my head as the most natural solution to the problem. But it did happen well before the programming blogosphere started making the pattern popular.
db.getUsers() I am sorry what? Who in their right mind loads all users from the database and then filters out the expired subscription ones. Shouldn't the database query do this?
I don't really like the example (and it's from Google) because, beyond the general concept, it seems like the trigger for sending emails is calling bulkSend with Date.now() instead of the user actually triggering an email when it's really expired: user.subscriptionEndDate change to < Date.now().
I like the general idea, but unless you're assuming some very clever language or even more clever ORM that fixes things implicitly, wouldn't
get all users and then filter out the few that will expire in 5 days, on a code level? That doesn't sound like it would scaleI agree. They could have picked a better example. Just db.getUsers() alone should set off alarm bells as soon as you see it.
Even large companies are still grasping at straws when it comes to good code. Meanwhile there are articles I wrote years ago which explain clearly from first principles why the correct philosophy is "Generic core, specific shell."
I actually remember early in my career working for a small engineering/manufacturing prototyping firm which did its own software, there was a senior developer there who didn't speak very good English but he kept insisting that the "Business layer" should be on top. How right he was. I couldn't imagine how much wisdom and experience was packed in such simple, malformed sentences. Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
"Generic core, specific shell."
Your advice is the opposite of "functional core, imperative shell". The FCIS principle has IS which is generic, to be simple, because it's usually hard to test (it deals with resources and external dependencies). So by being simple, it's more unit testable.
On the other hand, FC is where the business logic lives, which can be complex and specific. The reason why you want that "functional" (really just another name for "composable from small blocks") is because it can be tested for validity without external dependencies.
So the IS shields you from technicalities of external dependencies, like what kind of quirks your DB has, or are we sending data over network or writing to the file, or does the user inputs comands in spanish or english, do you display the green square or blue triangle to indicate the report is ready, etc.
On the other hand, FC deals with the actual business logic (what you want to do), which can be both generic and specific. These are just different types of building blocks (we call them functions) living in the FC.
FCIS is exemplified by user-shell interaction. The user (FC) dictates the commands and interprets the output, according to her "business needs". While the shell (IS) simply runs the commands, without any questions of their purpose. It's not the job of IS to verify or handle user errors caused by wrong commands.
But the user doesn't do stuff on her own; you could take her to a pub and she would tell you the same sequence of commands when facing the same situation. In that sense, the user is "functional" - independent on the actual state of the computer system, like the return value of a mathematical function is only dependent on the arguments.
Another example is MVC, where M is the FC and VC is the IS. Although it's not always exactly like that, for variety of reasons.
You can think of IS as a translator to a different language, understood by "the other systems", while the FC is there to supply what is actually being communicated.
I disagree that these two pieces of advice are opposed. I think they are orthogonal at worst, and in agreement at best.
"Functional core, imperative shell" (FCIS) is a matter of implementing individual software components that need to engage with side-effects --- that is, they have some impact on some external resources. Rather than threading representations of the external resources throughout the implementation, FCIS tells us to expel those concerns to the boundary. This makes the bulk of the component easier to reason about, being concerned with pure values and mere descriptions of effects, and minimizes the amount of code that must deal with actual effects (i.e. turning descriptions of effects into actual effects). It's a matter of comprehensibility and testability, which I'll clumsily categorize as "verification": "Does it do what it's supposed to do?"
"Generic core, specific shell" (GCSS) is a matter of addressing needs in context. The problems we need solved will shift over time; rather than throwing away a solution and re-solving the new problem from scratch, we'd prefer to only change the parts that need changing. GCSS tells us we shouldn't simply solve the one and only problem in front of us; we should use our eyes and ears and human brains to understand the context in which that problem exists. We should produce a generic core that can be applied to a family of related problems, and adapt that to our specific problem at any specific time using a, yes, specific shell. It's a matter of adaptability and solving the right problem, which I'll clumsily categorize as "validation": "Is what it's supposed to do what we actually need it to do?"
Ideally, GCSS is applied recursively: a specific shell may adapt an only slightly more generic core, which then decomposes into a smaller handful of problems that are themselves implemented with GCSS. When business needs change in a way that the outermost "generic core" can't cover, odds are still good that some (or all) of its components can still be applied in solving the new top-level problem. FCIS isn't really amenable to the same recursion.
Both verification and validation activities are necessary. One is a matter of internal consistency within the component; the other is a matter of external consistency relative to the context the component is being used in. FCIS and GCSS advise on how to address each concern in turn.
> I think they are orthogonal at worst, and in agreement at best.
I have considered them being orthogonal, but then the definition of the "shell" and "core" becomes problematic in this comparison. What you call shell in GCSS is not shell in FCIS at all, more like a boundary. Even there it is questionable whether boundary should be more specific than the core. At the core, things can be more integrated than at the boundary, and so it can have more business-specific rules.
I am not disputing GP's advice, but I feel it is perhaps a little bit simplistic if not tautological ("prefer generic building blocks where possible"), and really muddles up what the core and shell is in the FCIS meaning.
> the correct philosophy is "Generic core, specific shell."
> Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
I'm torn on this. This really is the faster way to higher quality.
OTOH, if more developers knew this, I wouldn't be so much more faster when I create my systems for clients. I'd just be a "normal 1x dev".
I like implementing features, sans AI-assistance, in my LoB applications faster than devs with Claude code doing so on their $FRAMEWORK.
> Even large companies are still grasping at straws when it comes to good code
Probably many reasons for this, but what I've seen often is that once the code base has been degraded, it's a slippery slope downhill after that.
Adding functionality often requires more hacks. The alternative is to fix the mess, but that's not part of the task at hand.
I've seen it many times. And then every task takes longer than the last one, which is what pushes teams to start rewrites. "There's never enough time to do it right, but always time to do it again."
> Probably many reasons for this, but what I've seen often is that once the code base has been degraded, it's a slippery slope downhill after that.
Another factor, and perhaps the key factor, is that contrary to OP's extraordinary claim there is no such thing as objectively good code, or one single and true way of writing good code.
The crispest definition of "good code" is that it's not obviously bad code from a specific point of view. But points of view are also subjective.
Take for example domain-driven design. There are a myriad of books claiming it's an effective way to generate "good code". However, DDD has a strong object-oriented core, to the extent it's nearly a purist OO approach. But here we are, seeing claims that the core must be functional.
If OP's strong opinion on "good code" is so clear and obvious, why are there such critical disagreements at such a fundamental levels? Is everyone in the world wrong, and OP is the poor martyr that is cursed with being the only soul in the whole world who even knows what "good code" is?
Let's face it: the reason there is no such thing as "good code" is that opinionated people making claims such as OP's are actually passing off "good code" claims as proxy's for their own subjective and unverified personal taste. In a room full of developers, if you throw a rock at a random direction you're bound to hit one or two of these messiahs, and neither of them agrees on what good code is.
Hearing people like OP comment on "good code" is like hearing people comment on how their regional cuisine is the true definition of "good food".
These are great and succinct, yours and your teammate’s.
I still find myself debating this internally, but one objective metric is how smoothly my longer PTOs go:
The only times I haven’t received a single emergency call were when I left teammates a a large and extremely specific set of shell scripts and/or executables that do exactly one thing. No configs, no args/opts (or ridiculously minimal), each named something like run-config-a-for-client-x-with-dataset-3.ps1 that took care of everything for one task I knew they’d need. Just double click this file when you get the new dataset, or clone/rename it and tweak line #8 if you need to run it for a new client, that kind of thing.
Looking inside the scripts/programs looks like the opposite of all of the DRY or any similar principles I’ve been taught (save for KISS and others similarly simplistic)
But the result speaks for itself. The further I go down that excessively basic path, the more people can get work done without me online, and I get to enjoy PTO. Anytime i make a slick flexible utility with pretty code and docs, I get the “any chance you could hop on?” text. Put the slick stuff in the core libraries and keep the executables dumb
I see a similar problem in infra-land where people expose too many config variables for too many things, creating more cruft. Knowing what to hardcode and what to expose as a var is something a lot of devs don't seem to understand; and don't realise they don't understand.
Yes I feel that when to apply certain techniques is frequently under-discussed. But I can't blame people for err-ing on the side of 'do everything properly' - as this makes life more pleasant in teams. Although I think if you squint, the principle still applies to your example. The further you get from the 'core' of your platform/application/business/what-have-you, the less abstract you need to be.
That is pretty convincing advice!! Maybe I’ll try writing more specific scripts.
I’d love to know more, do you have any links to your articles?
"Specific on the surface, generic underneath" (Medium paywalled): https://medium.com/tech-renaissance/generic-internals-specif...
> The more specific, the more brittle. The more general, the more stable. Concerns evolve/decay at different speeds, so do not couple across shearing layers. Notice how grammar/phonology (structure) changes slowly while vocabulary (functions, services) changes faster.
...
> Coupling across layers invites trouble (e.g. encoding business logic with “intuitive” names reflecting transient understanding). When requirements shift (features, regulations), library maintainers introduce breaking changes or new processor architectures appear, our stable foundations, complected with faster-moving parts, still crack!
https://alexalejandre.com/programming/coupling-language-and-...
Bertrand Meyer suggested another way to consider this that ends up in a similar place.
For concerns of code complexity and verification, code that asks a question and code that acts on the answers should be separated. Asking can be done as pure code, and if done as such, only ever needs unit tests. The doing is the imperative part, and it requires much slower tests that are much more expensive to evolve with your changing requirements and system design.
The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable, and they are easy to accidentally expose to third party code through the addition of subsequent features, even if initially they are unreachable. Sun biffed this way a couple of times with Java.
But for non crosscutting concerns this advice can also be a step toward FC/IS, both in structuring the code and acclimating devs to the paradigm. Because you can start extracting pure code sections in place.
Command-Query Separation is the term for that. However, I find this statement odd:
> having functions that do things without verifying preconditions are exploitable
Why would you do this? The separation between commands and queries does not mean that executing a command must succeed. It can still fail. Put queries inside the commands (but do not return the query results, that's the job of the query itself) and branch based on the results. After executing a command which may fail, you can follow it with a query to see if it succeeded and, if not, why not.
https://en.wikipedia.org/wiki/Command%E2%80%93query_separati...
I think CQRS is something different than what’s being described here. “Query” code in CQRS can still “do stuff”: call an external database, grab locks, audit trail recording etc.
What’s being described here is something lower level, that you keep as much code as you can as a side-effect-free “pure functional core”. That pattern is useful both for the “command” and “query” side of a CQRS system, and is not the same thing as CQRS
In asynchronous environments, you may not be able to repeat the same query with the same result (unless you control a cache of results, which has its own issues). If some condition is determined by the command’s implementation that subsequent code is interested in (a condition that isn’t preventing the command from succeeding), it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
I can’t decide if this really is the biggest problem with CQS. Certainly the wiki page claims it is, and it’s a reasonable argument. For some simpler cases you could dodge it by wrapping the function pairs/tuples in a lock. Database calls are a bit sketchy, because a transaction only “fixes” the problem if you ignore the elephant in the room which is reduced system parallelism by a measurable amount because even in an MVCC database transactions aren’t free. They’re just cheaper.
Caches always mess up computational models because they turn all reads into writes. Which makes things you could say with static analysis no longer true. I know a lot of tricks for making systems faster and I’ve hardly ever seen anyone apply most of them to systems after caching was introduced. It has one upside and dozens of downsides as bad or worse than this one.
> it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
You don't need the command to return anything (though it can be more efficient or convenient). It can set state indicating, "Hey, I was called but by the time I tried to do the thing the world and had changed and I couldn't. Try using a lock next time."
The caller can still obtain a result following the command, though it does mean the caller now has to explicitly retrieve a status rather than getting it in the return value.Where is that state stored, in an environment where the same command could be executed with the same parameters but resulting in a different status, possibly in parallel? How do you connect the particular command execution with the particular resulting status? And if you manage to do so, what is actually won over the command just returning the status?
I’d argue that the separation makes things worse here, because it creates additional hidden state.
Also, as I stated, this is not about error handling.
CQRS should really only guide you to designing separate query and command interfaces. If your processing is asynchronous then you have no choice but to have state about processing-in-flight, and your commands should return an acknowledgement of successful receipt of valid commands with a unique identifier for querying progress or results. If your processing is synchronous make your life easier by just returning the result. Purity of CQRS void-only commands is presentation fodder, not practicality.
(One might argue that all RPC is asynchronous; all such arguments eventually lead to message buses, at-least-once delivery, and the reply-queue pattern, but maybe that's also just presentation fodder.)
> Why would you do this?
Performance and re-use are two possible reasons.
You may have a command sub-routine that is used by multiple higher-level commands, or even called multiple times within by a higher-level command. If the validation lives in the subroutine, that validation will be called multiple times, even when it only needs to be called once.
So you are forced to choose either efficiency or the security of colocating validation, which makes it impossible to call the sub-routine with unvalidated input.
Perhaps I was unclear, to add to my comment:
hinkley poses this as a fault in CQS, but CQS does not require your commands to always succeed. Command-Query Separation means your queries return values, but produce no effects, and your commands produce effects, but return no values. Nothing in that requires you to have a command which always succeeds or commands which don't make use of queries (queries cannot make use of commands, though). So a better question than what I originally posed:
My "Why would you do this?" is better expanded to: Why would you use CQS in a way that makes your system less secure (or safe or whatever) when CQS doesn't actually require that?
The example in the wiki page is far more rudimentary than the ones I encountered when I was shown this concept. Trivial, in fact.
CQS will rely on composition to do any If A Then B work, rather than entangling the two. Nothing forces composition except information hiding. So if you get your interface wrong someone can skip over a query that is meant to short circuit the command. The constraint system in Eiffel I don’t think is up to providing that sort of protection on its own (and the examples I was given very much assumed not). Elixir’s might end up better, but not by a transformative degree. And it remains to be seen how legible that code will be seen as by posterity.
That's still not really answering my question for you, which was less clear than intended. To restate it:
> The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable
My understanding of your comment was that "this advice" is CQS. So you're saying that CQS commands do not verify preconditions and that this is a weakness in CQS, in particular.
Where did you get the idea that CQS commands don't verify preconditions? I've never seen anything in any discussion of it, including my (admittedly 20 years ago) study of Eiffel.
And I remain confused by your confusion.
If A then B()
Versus
B()
Somewhere there’s a B without the associated query. Call it what you want, at the bottom of the tree two roads diverge. Otherwise there is no Separation in your CQS.
ETA: once you get down to the mutation point you aren’t just dealing with immutable data. You’re moving things around, often plural.
I like the idea but the example doesn't make much sense.
In what application would you load all users into memory from database and then filter them with TypeScript functions? And that is the problem with the otherwise sound idea "Functional core, imperative shell". The shell penetrates the core.
Maybe some filters don't match the way database is laid out, what if you have a lot of users, how do you deal with email batching and error handing?
So you have to write the functional core with the side effect context in mind, for example using query builder or DSL that matches the database conventions. Then weave it with the intricacies of your email sender logic, maybe you want iterator over the right size batches of emails to send at once, can it send multiple batches in parallel?
I am surprised by this example, for the same reason.
Generally, performance is a top cause of abstraction leaks and the emergence of less-than-beautiful code. On an infinitely powerful machine it would be easy and advisable to program using neat abstracrions, using purely "the language of" the business. Our machines are not infinitely powerful, and that is especially evident when larger data sets are involved. That's where, to achieve useful performance, you have to increasingly speak "the language of" the machine. This is inevitable, and the big part of the programmer's skill is to be able to speak both "languages", to know when to speak which one, and produce readable code regardless.
Database programming is a prime example. There's a reason, for example, why ORMs are very messy and constitute such excellent footguns: they try to gap this bridge, but inevitably fail in important ways. And having and ORM in the example would, most likely, violate the "functional core" principle from the article.
So it looks like the author accidentally presented a very good counterexample to their own idea. I like the idea though, and I would love to know how to resolve the issue.
> In what application would you load all users into memory from database and then filter them with TypeScript functions?
You’d be surprised! I have worked on a legacy PHP service which did something very similar
I never liked encountering code that chains functions calls together like this
email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now())));
Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say.
I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
https://docs.pascal65.org/en/latest/langref/funcproc/
> email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now())));
What makes it hard to reason about is that your code is one-dimensional, you have functions like `getExpiredUsers` and `generateExpiryEmails` which could be expressed as composition of more general functions. Here is how I would have written it in JavaScript:
The idea is that you have small but general functions, methods and properties and then use higher-order functions and methods to compose them on the fly. This makes the code two-dimensional. The outer dimension (`filter` and `map`) tells the reader what is done (take all users, pick out only some, then turn each one into something else) while the outer dimension tells you how it is done. Note that there is no function `getExpiredUsers` that receives all users, instead there is a simple and more general `isExpired` method which is combined with `filter` to get the same result.In a functional language with pipes it could be written in an arguably even more elegant design:
I also like Python's generator expressions which can express `map` and `filter` as a single expression:I guess I just never encounter code like this in the big enterprise code bases I have had to weed through.
Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Do you just do the const emails =
three different times?
In my coding world it looks a lot like doing a SELECT * ON users WHERE isExpired < Date.now
but in some cases you just grab it all, loop through it all, and do little switches to do different things based on different isExpired.
But perhaps you can write a more generic function like generateExpiryEmailOrWhatever that understands the user object and contains the logic for what type of email to draft. It might need to output some flag if, for a particular user, there is no need to send an email. Then you could add a filter before the final (send) step.
since were just making up functions..
The whole pipeline thing is a red herring IMO.> Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data.... > > Do you just do the const emails = > > three different times?
If it's just two or three cases I might actually just copy-paste the entire thing. But let's assume we have twenty or so cases. I'll use Python notation because that's what I'm most familiar with. When I write `Callable[[T, U], V]` that means `(T, U) -> V`.
Let's first process one user at a time. We can define an enumeration for all our possible categories of user. Let's call this enumeration `UserCategory`. Then we can define a "categorization function" type which maps a user to its category:
I can then map each user to a tuple of category and user: Now I need a mapping from user category to processing function. I'll assume we call the processing function for side effects only and that it has no return value (`None` in Python): This mapping uses the user category to look up a function to apply to a user. We can now put it all together: map each user to a pair of the user's category and the user, then for each pair use the mapping to look up the processing function: OK, that's processing one user a time, but what if we want to process users in batches? Meaning I want to get all expired users first, and then send a message to all of them at once instead of one at a time. We can actually reuse most of our code because how how generic it is. The main difference is that instead of using `map` we want to use some sort of `group_by` function. There is `itertools.groupby` in the Python standard library, but it's not exactly what we need, so let's write our own: Now we can categorize our users into batches based on their category: To process these batches we need a mapping from batch to a function which process an iterable of users instead of just a single user. Now we can put it all together: There are quite a lot of small building block functions, and if all I was doing was sending emails to users it would not make sense to write these small function that add indirection. However, in a large application these small functions become generic building blocks that I can use in higher-order functions to define more concrete routines. The `group_by` function can be used for many other purposes with any type. The categorization function was used for both one-at-a-time and batch processing.I have been itching to write a functional programming book for Python. I don't mean a "here is how to do FP in Python" book, you don't need that, the documentation of the standard library is good enough. I mean a "learn how to think FP in general, and we are going to use Python because you probably already know it". Python is not a functional language, but it is good enough to teach the principles and there is value in doing things with "one hand tied behind your back". The biggest hurdle in the past to learning FP was that books normally teach FP in a functional language, so now the reader has to learn two completely new things.
Since everyone's giving !opinions, in my C# DDD world you'd ideally be able to:
That then writes the "send expiry email" commands from the aggregate, to an outbox, which a worker then picks up to send. Simple, transactional domain logic.In Elixir this would be written as:
I think Elixir hits the nail on the head when it comes to finding the right balance between functional and imperative style code.Not a single person in this thread commented on the use of Date.now() and similar - surely clock.now() - you never ever want to use global time in any code, how could you test it?
clock in this case is a thing that was supplied to the class or function. It could just be a function: () -> Instant.
(Setting a global mock clock is too evil, so don't suggest that!)
I was just referring to how pipes make these kinds of chained function calls more readable. But on your point, I think using Date.now() is perfectly ok.
The nice thing with the Elixir example is that you can easily `tap()` to inspect how the data looks at any point in the pipeline. You can also easily insert steps into the pipeline, or reuse pipeline steps. And due to the way modules are usually organized, it would more realistically read like this, if we were in a BulkEmails module:
The nice thing here is that we can easily log to the console, and also filter out nil expiry emails. In production code, `generate_expiry_email/1` would likely return a Result (a tuple of `{:ok, email}` or `{:error, reason}`), so we could complicate this a bit further and collect the errors to send to a logger, or to update some flag in the db.It just becomes so easy to incrementally add functionality here.
---
Quick syntax reference for anyone reading:
- Pipelines apply the previous result as the first argument of the next function
- The `/1` after a function name indicates the arity, since Elixir supports multiple dispatch
- `&fun/1` expands to `fn arg -> fun(arg) end`
- `&fun(&1, "something")` expands to `fn arg -> fun(arg, "something") end`
Not sure I like how the binding works for user in this example, but tbh, I don't really have any better idea.
Writing custom monad syntax is definitely quite a nice benefit of functional languages IMO.
> I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
What you want is to use a language that has higher-kinded types and monads so that functions can have both effects (even multiple distinct kinds of effects) and return values, but the distinction between the two is clear, and when composing effectful functions you have to be explicit about how they compose. (You can still say "run these three possibly-erroring functions in a pipeline and return either the successful result or an error from whichever one failed", but you have to make a deliberate choice to).
Making a distinction between pure and effectful functions doesnt require any kind of effect system though.
Having a language where "func" defines a pure function and "proc" defines a procedure that can performed arbitrary side effects (as in any imperative language really) would still be really useful, I think
> Having a language where "func" defines a pure function and "proc" defines a procedure that can performed arbitrary side effects (as in any imperative language really) would still be really useful, I think
Rust tried that in the early days, the problem is no-one can agree on exactly what side effects make a function non-pure. You pay almost all the costs of a full effect system (and even have to add an extra language keyword) but get only some of the benefits.
I may have gotten nerd sniped here, but I believe all of these examples so far have some subtle errors. Using elixir syntax, I would think something like this covers most of the cases:
Mainly a lot of these examples do the expiry filtering on the application side instead of the database side, and most would send expiry emails multiple times which may or may not be desired behavior, but definitely isn't the best behavior if you automatically rerun this job when it fails.----
Edit: I actually see a few problems with this, too, since Email.bulkSend probably shouldn't know about which user each email is for. I always see a small impedance mismatch with this sort of pipeline, since if we sent the emails individually it would be easy to wrap it in a small function that passes the user through on failure.
If I were going to build a user contacting system like this I would probably want a separate table tracking emails sent, and I think that the email generation could be made pure, the function which actually sends email should probably update a record including a unique email_type id and a date last sent, providing an interface like: `send_email(user_query, email_id, email_template_function)`
On the same page here, read it multiple times to see if I can convince my mind, this is bit off in terms of reading the code as its being executed. There are high chances of people making mistakes over the time with such patterns. As usual there is always a trade off involved, readability is the one taking hit here.
That's pretty hardcore, like you want to restrict the runtime substitution of function calls with their result values? Even Haskell doesn't go that far.
Generally you'd distinguish which function call introduces the error with the function call stack, which would include the location of each function's call-site, so maybe the "low-effort" label is accurate. But I could see a benefit in immediately knowing which functions are "pure" and "impure" in terms of manipulating non-local state. I don't think it changes any runtime behavior whatsoever, really, unless your runtime schedules function calls on an async queue and relies on the order in code for some reason.
My verdict is, "IDK", but worth investigating!
> you want to restrict the runtime substitution of function calls with their result values?
I don't get how you got there from parent comment.
Pascal just went with a needless syntax split of (side-effectful) methods and (side-effectful) functions.
It has been so long since I worked on the code that had chaining functions and caused problems that I am not sure I can do justice to describing the problems.
I vaguely remember the problem was one function returned a very structured array dealing with regex matches. But there was something wrong with the regex where once in a blue moon, it returned something odd.
So, the chained functions did not error. It just did something weird.
Whenever weird problems would pop up, it was always passed to me. And when I looked at it, I said, well...
I am going to rewrite this chain into steps and debug each return. Then run through many different scenarios and that was how I figured out the regex was not quite correct.
You could write the logic in a more straight forward, but less composable way, so that all the logic resides in one pure function. This way you can also keep the code to only loop over the users once.
email.sendBulk(generateExpiryEmails(db.getUsers(), Date.now()));
I would have written each statement on its own line:
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);
email.bulkSend(expiryEmails);
This is not only much easier to read, it's also easier to follow in a stack trace and it's easier to debug. IMO it's just flat out better unless you're code golfing.
I'd also combine the first two steps by creating a DB query that just gets expired users directly rather than fetching all users and filtering them in memory:
expiredUsers = db.getExpiredUsers(Date.now());
Now I'm probably mostly getting zero or a few users rather than thousands or millions.
(author here)
This is actually closer to the way the first draft of this article was written. Unfortunately, some readability was lost to make it fit on a single page. 100% agree that a statement like this is harder to reason about and should be broken up into multiple statements or chained to be on multiple lines.
Yeah. I did not mention what I would do, but what you wrote is pretty much what I prefer. I guess nobody likes it these days because it is old procedural style.
There's nothing procedural about binding return values to variables, so long as you aren't mutating them. Every functional language lets you do that. That's `let ... in` in Haskell.
Glad to see this. This style seems like it’s out of vogue now, but I find it much, much easier to reason about.
I agree because it reads as it will process in the direction I normally read. But I do think one of the benefits of the function approach is that the scope isn't cluttered with staging variables.
For these reasons one of the things I like to do in Swift is set up a function called ƒ that takes a single closure parameter. This is super minimal because Swift doesn't require parenthesis for the trailing closure. It allows me to do the above inline without cluttering the scope while also not increasing the amount of redirection using discrete function declarations would cause.
The above then just looks like this:
I would argue that the real key is to have a distinct core and shell, and to hold the core to a much higher standard of quality than the shell. In this article, being "functional" is just serving as a proxy for code quality.
> In this article, being "functional" is just serving as a proxy for code quality.
It is not, it is being very specific about what it means and what it is referring to
I wrote our AI agents code with a functional core + imperative shell and I have to agree: this approach yields much faster cycle times because you can run pure unit tests and it makes testing a lot easier.
We have tens of thousands of lines of code for the platform and millions of workflow runs through them with no production errors coming from the core agent runtime which manages workflow state, variables, rehydration (suspend + resume). All of the errors and fragility are at the imperative shell (usually integrations).
Some of the examples in this thread I think get it wrong.
This is already wrong because the call already starts with I/O; flip it and it makes a lot more sense.What you really want is (in TS, as an example):
The effect of this is that the inner logic of `bulkSend` is completely decoupled from I/O and external logic. Now there's no need for mocking or integration tests because it is possible to use pure unit tests by simply swapping out the functions. I can easily unit test `bulkSend` because I don't need to mock anything or know about the inner behavior.I chose this approach because writing integration tests with LLM calls would make the testing run too slowly (and costly!) so most of the interaction with the LLM is simply a function passed into our core where there's a lot of logic of parsing and moving variables and state around. You can see here that you no longer need mocks and no longer need to spy on calls because in the unit test, you can pass in whatever function you need and you can simply observe if the function was called correctly without a spy.
It is easier than most folks think to adopt -- even in imperative languages -- by simply getting comfortable working with functions at the interfaces of your core API. Wherever you have I/O or a parameter that would be obtained from I/O (database call), replace it with a function that returns the data instead. Now you can write a pure unit test by just passing in a function in the test.
I am very surprised how many of the devs on the team never write code that passes a function down.
I like the idea of separating your “business logic” (the functional core) from the glue code that interacts with the outside world (the imperative shell). It makes the core easier to test and reason about.
But also: the challenge is knowing where to draw the line. In real systems you’ll still have messy side-effects, transactions, performance constraints — so you might end up in a mixed bag anyway. The principle is solid, but the practical trade-offs matter.
The MirageOS project [0] is a great collection of functionality pure OCaml libraries that are useful outside of unikernels. I've used the DNS library with an effectful layer for various nameserver experiments [1].
[0] https://mirage.io/
[1] https://ryan.freumh.org/eon.html
I have written a small system in Elixir adhering to FCIS. Not used to the approach, I was pretty slow and sometimes it felt like jumping through hoops set by myself, lol, but I loved it, the code was very clean, testable, and refactorable. Highly recommend it as an exercise, it was surprising just how much state and IO can be pushed out.
Functions can have complexity or side effects, but not both.
All pure functions have complexity?
All nice ideas, that unfortunately don't get appreciated on the age of offshoring and vibe coding.
Have to ship it non matter what.
I think it's just your way of looking at things.
What if a FCF (functional core function) calls another FCF which calls another FCF? Or do we do we rule out such calls?
Object Orientation is only a skin-deep thing and it boils down to functions with call stack. The functions, in turn, boil down to a sequenced list of statements with IF and GOTO here and there. All that boils boils down to machine instructions.
So, at function level, it's all a tree of calls all the way down. Not just two layers of crust and core.
Functional core usually means pure functional functions, aka the return value is know if the arguments is known, no side effects required. All the side effects is then pushed up the imperative shell.
You’ll find usually that side effect in imperative actions is usually tied to the dependencies (database, storage, ui, network connections). It can be quite easy to isolate those dependencies then.
It’s ok to have several layers of core. But usually, it’s quite easy to have the actual dependency tree with interfaces and have the implementation as leaves for each node. But the actual benefits is very easy testing and validation. Also fast feedback due to only unit tests is needed for your business logic.
If your language supports generators, this works a lot better than making copies of the entire dataset too.
Sometimes, sure - but sometimes, passing around a fat wrapper around a DB cursor is worse, and the code would be better off paginating and materializing each page of data in memory. As usual, it depends.
You don't need your programming language to implement generators for you. You can implement them yourself.
Another good blog post that is IMO in the same vein: https://lambdaisland.com/blog/2022-03-10-mechanism-vs-policy (“Improve your code by separating mechanism from policy”). This blends harmoniously with “functional core, imperative shell”—the "mechanism" code is the "functional core", and the "policy" code is the "imperative shell"—and also a little bit with John Ousterhout's idea in A Philosophy of Software Design of "deep modules" (in this context, don't put policy stuff, i.e. arbitrary decisions, inside the module).
Reminds me of this clean architecture talk with Python explains this very well: https://www.youtube.com/watch?v=DJtef410XaM
Haskell practically encourages this style of programming. Any function that touches IO needs to wrap outputs with an appropriate monad. It becomes easier to push all IO out to the edges of your program and keep your core purely functional with no monads
I wish that's what people did, some codebases I've seen are messes of monad transformer stacks the likes of which you've never seen.
I mean, what if you want to do IO and have mutable data structures inside a do block? I'm afraid I'm going to have to prescribe you a monad transformer. Be careful of the side effects.
[dead]
This works right up to the point where you try to make the code to support opening transactions functional. :D
Some things are flat out imperative in nature. Open/close/acquire/release all come to mind. Yes, the RAI pattern is nice. But it seems to imply the opposite? Functional shell over an imperative core. Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
Edit: I certainly think having some sort of affordance in place to indicate if you are in different sections is nice.
> This works right up to the point where you try to make the code to support opening transactions
Indeed. It's all well and good to impart some kind of flavour into your code and call it functional, but transactions do not give a crap about style.
A transaction needs to be able to 'back out' to fulfill 'all-or-nothing' semantics. Side effects are what make this impossible.
whispers in monads
It can be done "functionally" but doesn't necessarily have to be done in an FP paradigm to use this pattern.
There are other strategies to push resource handling to the edges of the program: pools, allocators, etc.
Right, but even in those, you typically have the more imperative operations as the lower levels, no? Especially when you have things where the life cycle of what you are starting is longer than the life cycle of the code that you use to do it?
Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
> but even in those, you typically have the more imperative operations as the lower levels
Yes, the monadic part is the functional core, and the runtime is the imperative shell.
> Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
What do you mean by Monad trick? That's precisely the kind of thing the IO monad exists for. If you need to fetch things on an API: IO. If you need to read/save things on a DB: IO. DB Transaction: IO.
I have not seen too many (any?) times where the monad trick is done in such a way that they don't combine everything in a single context wrapper and talk about the "abnormal" case where things don't complete during execution.
Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)
You would deal with this problem in the same way you would with, say, in a REST API.
If the transaction object is serializable you can just store it in a DB, for example. If it's some C++ pointer from some 3rd-party library that you can't really serialize and gotta keep open, you gotta keep it in memory and manage its lifetime explicitly, be it a REST web server, in Haskell or in a C++ app.
I'm unclear what you're suggesting here. Are you suggesting you couldn't write a POS in Haskell, say?
My idea here is that, in many domains, you will have operations that are somewhat definitionally in the imperative camp. OpenTransaction being the easy example.
Can you implement it using functional code? Yes. Just make sure you wind up with partial states. And often times you are best off explicitly not using the RAI pattern for some of these. (I have rarely seen examples where they deal with this. Creating and reconciling transactions often have to be separate pieces of code. And the reconcile code cannot, necessarily, fallback to create a transaction if they get a "not found" fault.)
> Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
That's not what functional core, imperative shell means though. It's a given that CPUs aren't functional. The advice is for people programming in languages that have expressions - ruby, in the case of the original talk. The functional paradigm mostly assumes automatic memory management.
Right, I was just using that as "at the extreme" and how it largely exists to allow you to put a functional feel on top of the imperative below it.
I'm sympathetic to the idea, as you can see it in most instruction manuals that people are likely to consume. The vast majority of which (all of them?) are imperative in nature. There is something about the "for the humans" layer being imperative. Step by step, if you will.
I don't know that it fully works, though. I do think you are well served being consistent in how you layer something. Where all code at a given layer should probably stick to the same styles. But knowing which should be the outer and which the inner? I'm not clear that we have to pick, here. Feel free to have more than two layers. :D
This is same idea with onion architect in "Grokking Simplicity: Taming Complex Software with Functional Thinking Book by Eric Normand"
this looks like a post from 2007 im shocked at the date
And "I call it my billion-dollar mistake. It was the invention of the null reference in 1965" is from 2009.
Hopefully by 2045 these ideas will have gotten a little more traction.
Yeah, it’s based on an old post: https://www.destroyallsoftware.com/screencasts/catalog/funct...
I saw Gary posted his blog link on twitter, and I really like his article. I really didn't expect it to surface up at this moment (2025), and it's referred from a google blog. :shrug:
Me too. Aren't we already doing this? This is the basic I have been taught first.
Something like that was popular in perl world: functional core, oop external interface.
I invented this pattern when I was working on a small ecommerce system (written in Scheme, yay!) in the early 2000s. It just became much easier to do all the pricing calculations, which were subject to market conditions and customer choices, if I broke it up into steps and verified each step as a side-effect-free, data-in-data-out function.
Of course by "invented" I mean that far smarter people than me probably invented it far earlier, kinda like how I "invented" intrusive linked lists in my mid-teens to manage the set of sprites for a game. The idea came from my head as the most natural solution to the problem. But it did happen well before the programming blogosphere started making the pattern popular.
db.getUsers() I am sorry what? Who in their right mind loads all users from the database and then filters out the expired subscription ones. Shouldn't the database query do this?
I don't really like the example (and it's from Google) because, beyond the general concept, it seems like the trigger for sending emails is calling bulkSend with Date.now() instead of the user actually triggering an email when it's really expired: user.subscriptionEndDate change to < Date.now().
The for-loop is just better.
destory all software
that's nice, so should I get all the db users and then filter them in app?
Probably. Or better yet move the code to run where the data is so you're not moving the data around.
One of the core design principles at https://github.com/aperoc/toolkami
Spam
Not sure man, I specifically stated this in my README way before this post: https://github.com/aperoc/toolkami/blob/main/README.md#comma....
I mean it's not much, but the concept just resonates with me and I want to share it. Sad I can't share even simple opinion nowadays ...