To start I created the FibonacciNumber struct, and started implementing FromStr. I struggled to implement the Err for the Result type at first, because I was trying to combine an IntErrorKind::PosOverflow (to be returned for numbers greater than 100) and a ParseIntError. I couldn’t figure out how to limit my error enum to just the PosOverflow variant, the compiler kept telling me to use IntErrorKind. Eventually I gave up and just created my own variants. But once I had the enum it needed to be a proper std::error::Error which required implementing Debug (easy, because I could derive it) and Display (more toil but an easy enough pattern to follow).

I got momentarily distracted by the power of Display and implemented one for FibonacciNumber that gave each number the proper ordinal indicator. This was very satisfying.

Eventually I remembered the goal was FromStr and finished implementing that, returning either a Ok(FibonacciNumber) or an Err(FibonacciNumberError) in each branch.

The only nuance about using the FibonacciNumber was that I had to update my fibonacci function to borrow the number instead of moving it. I’m still learning about ownership in Rust so I’m not going to comment further on that other than to say: the compiler told me exactly what to do and I listened to it!

As previously: constructive feedback is much appreciated!

 1use std::{fmt::Display, io, str::FromStr};
 2
 3#[derive(Debug)]
 4enum FibonacciNumberError {
 5    TooBigError,
 6    ParseError,
 7}
 8impl Display for FibonacciNumberError {
 9    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
10        match self {
11            FibonacciNumberError::TooBigError => {
12                write!(f, "Number must be a positive integer less than 101.")
13            }
14            FibonacciNumberError::ParseError => write!(f, "Number must be a positive integer."),
15        }
16    }
17}
18impl std::error::Error for FibonacciNumberError {}
19
20struct FibonacciNumber {
21    n: u8,
22}
23impl Display for FibonacciNumber {
24    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
25        if [11, 12, 13].contains(&self.n) {
26            write!(f, "{}th", self.n)
27        } else {
28            match self.n % 10 {
29                1 => write!(f, "{}st", self.n),
30                2 => write!(f, "{}nd", self.n),
31                3 => write!(f, "{}rd", self.n),
32                _ => write!(f, "{}th", self.n),
33            }
34        }
35    }
36}
37impl FromStr for FibonacciNumber {
38    type Err = FibonacciNumberError;
39    fn from_str(s: &str) -> Result<Self, Self::Err> {
40        match s.trim().parse() {
41            Ok(num) => {
42                if num > 100 {
43                    Err(FibonacciNumberError::TooBigError)
44                } else {
45                    Ok(FibonacciNumber { n: num })
46                }
47            }
48            Err(_) => Err(FibonacciNumberError::ParseError),
49        }
50    }
51}
52
53fn main() {
54    println!("Enter fibonacci number to generate.");
55    let n: FibonacciNumber = loop {
56        let mut string_n = String::new();
57        io::stdin()
58            .read_line(&mut string_n)
59            .expect("Failed to read number!");
60
61        match string_n.parse() {
62            Ok(num) => break num,
63            Err(e) => {
64                println!("{e}");
65                continue;
66            }
67        }
68    };
69
70    let generated = fibonacci(&n);
71    println!("The {n} fibonacci number is {generated}.")
72}
73
74fn fibonacci(fib_num: &FibonacciNumber) -> u128 {
75    match fib_num {
76        FibonacciNumber { n: 0 } => 0,
77        _ => {
78            let mut minus2: u128 = 0;
79            let mut minus1: u128 = 1;
80            let mut maybe_nth: u128 = 1;
81            let mut counter = fib_num.n;
82            while counter > 1 {
83                maybe_nth = minus2 + minus1;
84                minus2 = minus1;
85                minus1 = maybe_nth;
86                counter -= 1;
87            }
88            maybe_nth
89        }
90    }
91}

The actual Fibonacci implementation feels a bit clunky, if only because I’m so used to reaching for tail recursion and I don’t think I have that in Rust (there is maybe a crate, but a lot of what I was reading was a few years old and the current state was unclear and I got overwhelmed).

If anyone is so moved: I am very much a beginner on my Rust journey, and any tips towards idiomatic Rust would be welcome.

 1use std::io;
 2
 3fn main() {
 4    println!("Enter fibonacci number to generate.");
 5    let n: u8 = loop {
 6        let mut string_n = String::new();
 7        io::stdin()
 8            .read_line(&mut string_n)
 9            .expect("Failed to read number!");
10
11        match string_n.trim().parse() {
12            Ok(num) => {
13                if num > 100 {
14                    println!("Please enter a positive integer less than 101.");
15                    continue;
16                } else {
17                    break num;
18                }
19            }
20            Err(_) => {
21                println!("Please enter a positive integer.");
22                continue;
23            }
24        };
25    };
26
27    let generated = match n {
28        0 => 0,
29        1 => 1,
30        _ => fibonacci(n),
31    };
32
33    println!("The {n}th fibonacci number is {generated}!")
34}
35
36fn fibonacci(n: u8) -> u128 {
37    let mut minus2: u128 = 0;
38    let mut minus1: u128 = 1;
39    let mut maybe_nth: u128 = 1;
40    let mut counter = n;
41    while counter > 1 {
42        maybe_nth = minus2 + minus1;
43        minus2 = minus1;
44        minus1 = maybe_nth;
45        counter -= 1;
46        println!("debug counter: {counter}, maybe_nth: {maybe_nth}")
47    }
48    maybe_nth
49}

This is something that comes up infrequently enough for me that I always remember it’s possible but infuriatingly can NEVER remember how to do it. It’s also really difficult to search for, so I spend a while feeling very frustrated with sbt, the sbt plugin that has made me ask the question, and myself.

I’m writing it down here in the hopes that I will remember I wrote it down, and have just one place on the internet to search!

The Steps:

1. In an sbt console, run reload plugins to get into the root project’s project/ build definition.
2. Once in the project build, you can run a lot of your usual commands. For this particular problem, I run dependencyBrowseTree (from sbt-dependency-tree which I have defined as a global plugin) to open a searchable tree of all project dependencies.
3. Once finished, reload return switches you back to the original build.

Pretty straightforward, just difficult to remember!

 1---
 2config:
 3  theme: base
 4  themeVariables:
 5    textColor: "#777"
 6    primaryColor: "#d33682"
 7    loopTextColor: "#6c71c4"
 8    labelBoxBkgColor: "#6c71c4"
 9    labelBoxBorderColor: "#6c71c4"
10  fontSize: 22
11---
12sequenceDiagram
13    participant A
14    participant B
15    participant C
16    A-->>+B: request to B
17    B-->>-A: response from B
18    A-->>+C: <br/><br/>request to C
19    C-->>-A: response from C
20    par many requests in parallel
21    note right of B: remember!
22    B-->>+C: request to C
23    C-->>-B: response from C
24    end

Notes about this config, in no particular order

• I couldn’t get background to work on GitHub to set a specific colour for the display window
  • This is why I had to set the font colour to a midtone grey, to try to support both a light and dark background
• the <br/><br/> adds some visual space, so that a request and its response are clustered together compared to distinct requests
• the par box & border are controlled by the label variables, the action text is controlled by the loop text colour
• fontSize is changed at the top level, NOT inside the theme variables

Screenshots

This is what the above sequence diagram looks like in dark mode and light mode on GitHub:

2024

• https://www.geoffreylitt.com/2024/08/25/your-pie-doesnt-need-be-original
• https://www.ursulakleguin.com/a-rant-about-technology
• https://citiesandmemory.com/obsolete-sounds/

The Problem

We have a stream of “things”. We want to batch those things up, and do something to each batch. We need to count which batch we’re on. Oh and also, if the stream of “things” is empty, we want to fallback to some other operation.

The Code

 1import cats.effect.IO
 2import fs2.Stream
 3
 4def getAndProcessThings(batchSize: Int): IO[Unit] = {
 5  val things: Stream[IO, String] = Stream("a", "b", "c", "d", "e", "f", "g", "h", "i", "j").covary[IO]
 6  val fallback: Stream[IO, Unit] = Stream.eval(IO.println("falling back"))
 7
 8  things
 9    .chunkN(batchSize) // split stream into chunks of batchSize
10    .zipWithIndex // zip the chunks together with an index
11    .evalMap { case (chunk, batchNumber) =>
12      IO.println(s"Processing batch $batchNumber: $chunk")
13    }
14    .ifEmpty(fallback)
15    .compile
16    .drain
17}

If that function is run with a batch size of 3, we’ll get the following output

1processing batch number 0; Chunk(a, b, c)
2processing batch number 1; Chunk(d, e, f)
3processing batch number 2; Chunk(g, h, i)
4processing batch number 3; Chunk(j)

If things was instead set to a value of Stream.empty, we’ll instead see falling back printed.

Blerf Inc. is also a company! You can hire me for various software engineering tasks, particularly where Scala is concerned. Reach out at admin@blerf.ca to discuss.

The Problem, A Recap

From the first post:

We have some data, rows of recommendations. The data is stored on disk, and we want to write an API that will return rows based on some query. The data, for now, is small enough that it can be read into memory when we start the server. A different process will update the data on disk however, and we need that update to propagate to the server without disrupting running requests. The update is the piece I’ve been working on.

I’m continuing from last time by focusing now on the API:

• how to use the Ref to construct a response?
• how to update the Ref via a request?
• how to link the http4s Server with the scheduled updates?

The Pieces

The method to update the Ref needed a few changes. The prog to do the actual updating is good; however instead of running mulitple streams like in the previous example, we’ll be using multiple resources. So now I’ve decided to compile the stream to a resource and return a Resource[IO, Unit].

1def scheduledUpdater(ref: Ref[IO, Int]): Resource[IO, Unit] = {
2  val prog = ref
3    .updateAndGet(i => i + 1)
4    .flatMap(i => IO.println(s"scheduled update: added 1, ref value is now $i"))
5  Stream.repeatEval(prog).metered(1.minute).compile.resource.drain
6}

Instead of just printing the Ref value on a schedule, I wanted to make a curl request for the current value. I also wanted to be able to set the value of the Ref arbitrarily via a request as well. For that I wrote a 2-endpoint service:

 1def refService(ref: Ref[IO, Int]) = HttpRoutes.of[IO] {
 2  case GET -> Root / "ref" =>
 3    for {
 4      i    <- ref.get
 5      resp <- Ok(s"Current ref value is: $i")
 6    } yield resp
 7  case PUT -> Root / "ref" / IntVar(value) =>
 8    for {
 9      _    <- ref.set(value)
10      resp <- Ok(s"Updated the ref value to: $value")
11    } yield resp
12}

Finally, similar to last time, the tricky part was figuring out how to stitch the scheduled update together with something else (in this case the Server resource). The real MVP here is the call to Resource.both, which “allocates two resources concurrently, and combines their results in a tuple.”¹ Then I can call .useForever on the tuple to have a forever-running server and updater.

 1def run(args: List[String]): IO[ExitCode] = {
 2  val serverAndUpdater: Resource[IO, (Server, Unit)] = for {
 3    ref <- Resource.eval(Ref[IO].of(1))
 4    httpApp = Router("/" -> refService(ref)).orNotFound
 5    updater = scheduledUpdater(ref)
 6    server = EmberServerBuilder
 7      .default[IO]
 8      .withHost(ipv4"0.0.0.0")
 9      .withPort(port"8080")
10      .withHttpApp(httpApp)
11      .build
12    serverAndUpdater <- Resource.both(server, updater)
13  } yield serverAndUpdater
14
15  serverAndUpdater.useForever.as(ExitCode.Success)
16}

All The Code Together

All the pieces come together like so:²

 1//> using lib "co.fs2::fs2-core:3.4.0"
 2//> using lib "org.typelevel::cats-effect:3.4.4"
 3//
 4//> using lib "org.http4s::http4s-core:0.23.17"
 5//> using lib "org.http4s::http4s-dsl:0.23.17"
 6//> using lib "org.http4s::http4s-ember-server:0.23.17"
 7//> using lib "org.http4s::http4s-server:0.23.17"
 8
 9import fs2.Stream
10import cats.effect.IO
11import cats.effect.IOApp
12import scala.concurrent.duration._
13import cats.effect.kernel.Ref
14
15import org.http4s.HttpRoutes
16import org.http4s.dsl.io._
17import org.http4s.implicits._
18import org.http4s.server.Router
19import org.http4s.ember.server.EmberServerBuilder
20
21import com.comcast.ip4s._
22import cats.effect.ExitCode
23import cats.effect.kernel.Resource
24import org.http4s.server.Server
25
26object ScheduledEndpoint extends IOApp {
27
28  def refService(ref: Ref[IO, Int]) = HttpRoutes.of[IO] {
29    case GET -> Root / "ref" =>
30      for {
31        i    <- ref.get
32        resp <- Ok(s"Current ref value is: $i")
33      } yield resp
34    case PUT -> Root / "ref" / IntVar(value) =>
35      for {
36        _    <- ref.set(value)
37        resp <- Ok(s"Updated the ref value to: $value")
38      } yield resp
39  }
40
41  def scheduledUpdater(ref: Ref[IO, Int]): Resource[IO, Unit] = {
42    val prog = ref
43      .updateAndGet(i => i + 1)
44      .flatMap(i => IO.println(s"scheduled update: added 1, ref value is now $i"))
45    Stream.repeatEval(prog).metered(1.minute).compile.resource.drain
46  }
47
48  def run(args: List[String]): IO[ExitCode] = {
49    val serverAndUpdater: Resource[IO, (Server, Unit)] = for {
50      ref <- Resource.eval(Ref[IO].of(1))
51      httpApp = Router("/" -> refService(ref)).orNotFound
52      updater = scheduledUpdater(ref)
53      server = EmberServerBuilder
54        .default[IO]
55        .withHost(ipv4"0.0.0.0")
56        .withPort(port"8080")
57        .withHttpApp(httpApp)
58        .build
59      serverAndUpdater <- Resource.both(server, updater)
60    } yield serverAndUpdater
61
62    serverAndUpdater.useForever.as(ExitCode.Success)
63  }
64
65}

And with that running, I can curl for the current value of the ref:

1❯ curl 0.0.0.0:8080/ref
2Current ref value is: 1%

Letting it sit and run for a while, I can see the ref getting updated:

1scheduled update: added 1, ref value is now 2
2scheduled update: added 1, ref value is now 3

I can get the updated ref value, and PUT a brand new value too:

1❯ curl 0.0.0.0:8080/ref
2Current ref value is: 3%
3
4❯ curl -X PUT 0.0.0.0:8080/ref/42
5Updated the ref value to: 42%
6
7❯ curl 0.0.0.0:8080/ref
8Current ref value is: 42%

Neat! At $work, the update function was a lot more complicated because it was a streaming request to read from a GCS bucket (and a lot of error handling to go with it), and the lookup was more complicated because we have more than just an integer. BUT the bones of this solution are what I’ve shipped and it’s going pretty well (touch wood, throw some salt over your left shoulder, etc. etc.).

The Problem

We have some data, rows of recommendations. The data is stored on disk, and we want to write an API that will return rows based on some query. The data, for now, is small enough that it can be read into memory when we start the server. A different process will update the data on disk however, and we need that update to propagate to the server without disrupting running requests. The update is the piece I’ve been working on.

The Code

I took this as an opportunity to build my understanding of Cats Effect Ref, since it “provides safe concurrent access and modification of its content.” I also wanted to use an fs2 Stream to handle the scheduling task (mostly because I know the fs2 stream API reasonably well and I could see how to do it. I’m not sure if there is a better way to do that piece.).

To start I defined a “simpler” version of my problem:

• suppose I have a single Ref, in IO, with an integer value
• I want to print the value every second
• I want to add 1 to the value every 3 seconds

The implementation¹ for this reduced problem looks like so:

 1//> using lib "co.fs2::fs2-core:3.4.0"
 2//> using lib "org.typelevel::cats-effect:3.4.4"
 3
 4import fs2.Stream
 5import cats.effect.IO
 6import cats.effect.IOApp
 7import scala.concurrent.duration._
 8import cats.effect.kernel.Ref
 9
10object Scheduled extends IOApp.Simple {
11  val theRefThing: IO[Ref[IO, Int]] = Ref[IO].of(1)
12
13  def scheduledUpdate(ref: Ref[IO, Int]): Stream[IO, Unit] = {
14    val prog = ref
15      .updateAndGet(int => int + 1)
16      .flatMap(int => IO.println(s"updated the ref to $int"))
17    Stream.repeatEval(prog).metered(3.seconds)
18  }
19
20  def printRefThing(ref: Ref[IO, Int]): Stream[IO, Unit] = {
21    val prog = ref.get
22      .flatMap(int => IO.println(s"the current ref value is ${int}"))
23    Stream.repeatEval(prog).metered(1.second)
24  }
25
26  def run: IO[Unit] = theRefThing.flatMap(ref =>
27    printRefThing(ref)
28      .concurrently(scheduledUpdate(ref))
29      .interruptAfter(10.seconds)
30      .compile
31      .drain
32  )
33}

The scheduledUpdate and printRefThing methods both take in the ref as an argument. The scheduling (printing every 1 second, or updating every 3 seconds) is handled by metering the streams.

The entire experiment is orchestrated together in the run method, by creating theRefThing once, and passing that same value to both methods. Those methods are run concurrently and interrupted after 10 seconds.

I emphasize once because in the moment (and in the haze of the holidays²) I struggled here. If I passed that initial IO around directly, then both streams would create a Ref each time their respective programs ran. Both methods needs to be using the same Ref for any of this to work.

The next step, which I’ll write up separately, expands this reduced problem to include the API piece (using http4s).

• is this post getting generated?
• how to make a footnote?
• What do these checkboxes look like?
  • they look totally reasonable
• Did I figure out syntax highlighting?
  • hot damn I did! Good job Past Sam!
• different code styling?
• line numbers??

I am looking at this on a small screen and it looks not great. I think I would like to experiment with different themes that are better layouts on a small screen.

1// this is just some sample code to check highlighting
2def blerf(name: String): Unit = println(s"Hey, ${name}, did you know that Sam blerfed it?")

To remember in the future

• If the netlify preview build fails but everything works locally, check the hugo version
• not all chromastyles are available it seems?
• the leading slashes for the logo paths are important

Hugo Things

It seems that the internet assumes I know things about Hugo while learning about Netlify. Obviously that is not the case so here are some notes about what some of the suggested hugo commands and flags mean.

--gc

Enables some cleanup tasks that will run after the build. I’m assuming it stands for garbage collection.

--minify

Enables minification of assets. Docs and other resources don’t exactly describe what “minification” means so I’m relying on my onomatopoetic interpretation here. I’m also currently assuming that there aren’t any technical choices to make about how assets are “minified”.

--buildFuture

Includes scheduled content (scheduling == setting a publish date to some time in the future?) in the build. I don’t foresee ever being organized enough to schedule things but sure.

--enableGitInfo

Adds git rev, date, and author info to the pages. In my reading, I’ve seen this flag specified only on the build command in the context.split1 section of the netlify.toml file. My current un-educated best guess is that this is related to split testing, so that git info is included on the test pages? I highly doubt I’ll be doing any kind of split testing so I don’t know if I really need this? But also every example I’ve looked at includes it sooooo maybe I’m wrong and it’s important. I won’t delete but I don’t like it.

Netlify Things

Most of Netlify configuration happens in the Netlify web gui (I think????). I guess I’m about to find out aren’t I?

$DEPLOY_PRIME_URL

Used in netlify.toml, references whatever is set as the primary domain in the site settings on Netlify. I believe that this parameter is only available within the netlify.toml file, it can’t be referenced all willy-nilly.

why does my deployed site look bad but my preview looks good?

This gave me a devil of a time, and I still don’t understand why the deploy-preview worked but the production deploy didn’t. There are some things I did that probably weren’t the problem but are good to do anyway that I’ve listed here. Ultimately I think the thing that kicked the system into gear was the CORS headers, but I’m not sure.

1. make sure the baseUrl in the config.toml matches the primary domain in Netlify
2. make sure the baseUrl in the config.toml uses https or else a lot of things get mad about mixed content
3. add a redirect from / to /index.html in netlify.toml
4. add a headers block to enable CORS