Rust for Java developers, part 2

In the first installment of this now series of Rust for Java developers, we saw some basic syntax, some pattern matching, briefly touched on enum and, most importantly, tried to reason about where the data in our programs resides.

When we quickly tried to save a line of code, we inadvertently changed the scope of an important resource because of RAII and the borrow checker gave us an error when we tried compiling the code:

The dreaded borrow checker with borrow later used here is effectively telling us a reference (a borrow) to a value owned by, in this case, a temporary is used while the owner was freed. This is an error message too well known to developers trying out Rust coming from managed languages. Chances are that if you started playing with Rust yourself, you’ve came across such a message sooner or later on your adventure.

Java not only has the garbage collector to free memory only when it isn’t used, it also has the compiler that can optimize heap allocations away entirely transparently. This frees you from thinking about memory, about freeing it or where it is allocated: on the heap or the stack. Rust, on the other hand, pushes that decision of where and how to allocate memory on the developer. Which comes with great power, but also… yeah, responsibility. At least the compiler will not let you produce unsafe code. Which sometimes makes it feel as if the compiler won’t let you get your job done. This generally comes from the coding patterns not accounting for how the data "flows" through your program, because the JVM has spoiled us in not having to think about those things and create a large sea of object where any one can point to, well…​ everything. So what does Rust do?

We’ll start from where we left off in the previous post, i.e. a brand new Rust project, as created with cargo new. We’ll change the code a bit, in not too many surprising ways again, to get the first argument passed, if any, and greet:

Wait a second! This is nothing like we’ve seen previously? Why is all we did before now a one-liner? Well… yes. We could have made it easier to read. Or at least I hope this is easier to read. All we need now to get the second, .nth(1), item from the Args iterator, which still is an Option<String>; we then .unwrap to move the String out of the Option if there was Some argument provided to our program. On the other hand, _or if there was None, we create a heap allocated String instance from the argument provided to the method: "World": &str instead.

There is one new bit of syntax in this code above: String::from(), which is a function from() invoked on the String type. This would be the equivalent of a static method call. The from() we refer to here is to be found on the struct String. A struct holds related data in named fields, like a class in Java does. But we’ll get back to that in a bit.

So what’s this business with moving the String out of the Option::Some? As we saw in the previous post, the String has to live somewhere. That’s what Rust calls ownership. And there can only be one owner of any piece of data. If there was an argument passed in to our program, the call to env::args().nth(1) would return that String in an Option::Some variant, which is the owner of the said String. Which means that as soon as it goes out of scope, its content would be Drop -ped as well and the String would be freed. But we don’t want to deal with the Option indirection throughout our little program. We only care about having a String, whether it’s the argument passed in, or "World", we don’t really care beyond getting who to point the proper String for us to print. Which is what this .unwrap_or() does for us.

Let’s break the two individual bits down, first we’ll only deal with the Option<String> for simplicity’s sake, so let’s refactor our code to first get that:

Now that we have our anybody_at_all: Option<String>, let’s assume that there is always an argument provided to our program. We can replace the .unwrap_or method call, by a straight .unwrap(), like this:

This code would now panic in the absence of an argument actually being provided. Let’s try this out, first providing an argument, then omitting it:

Right, so that’s bad, but let’s ignore this for now, we’ll get back to our .unwrap_or version soon enough. Focusing on the path that actually succeeds, calling anybody_at_all.unwrap() transfers the ownership of the String it had to the who: String variable. The String from the argument provided, i.e. "Jane" in this example, moved from the Option::Some that was our anybody_at_all to who. At which point you probably wonder what happened to anybody_at_all and you’d be right asking. Trial and error remains our approach here again: let’s try to .unwrap() the value again:

What would _again be? (starting a variable name with underscore tells the compiler we don’t plan on using it). It’s not a reference. It is also a String, as that’s what the Option contains. So would it be a copy of the original string? Let’s try and run this:

Ok, that’s a scary error message: move occurs because …​ does not implement the Copy trait?! Well, no copy of the original String apparently. So much on that hypothesis. But we do see something did move! anybody_at_all moved says the error message! The Option itself did. Because of some fn unwrap(self) method we did indeed invoke. We’ll get back to the syntax later. But what happened is that the call did result in anybody_at_all being consumed, while it transferred ownership of the String it contained to who. So there is nothing we can do with that anybody_at_all, it "vanished" and can’t be used beyond line 5, where we did invoke .unwrap().

Ownership is transferred from one variable to another within the same scope in a very similar fashion:

This would not compile, as we transfer the ownership of our String from the who variable to the moved one. As such who can not be used there after and println! can’t make use of it anymore.

What about the .unwrap_or(String::from("World")) call? It behaves exactly the same, other than it doesn’t panic if we’re dealing with a Option::None variant. In that case, it returns the argument provided instead. This is a generic type Option<T> and its unwrap_or takes a T as its default: T argument. Which means we have to pass it a String in this case. That’s why we have to instantiate a string on the heap, unlike what we did in the previous post and where we dealt with an &str instead. This program allocates a string on the heap for "World" always. This is an argument passed in to a method, and as such needs to be evaluated, which means the function String::from is always invoked. There is an alternative, running cargo clippy will actually recommend you use it, which makes use of Rust’s equivalent of Java’s lambda expression to lazily evaluate the default String to use and avoids unnecessarily allocate "World" on the heap if an argument was already provided. We’ll keep that for a later post though - feel free to try it though!

Now that we have a fairly good understanding of ownership and move, it’s time we look a little closer at borrowing. As we said quickly earlier a reference is a borrow of a value. We’ve been borrowing a fair amount in our previous post, as we dealt with a reference to a str from the loaded binary code, or a reference to the String that we got from the arguments passed to the executable. All those borrows were "read-only" ones. Meaning that we couldn’t ever mutate what the reference was pointing to. They come in opposition to &mut references, which are mutable references.

Here comes an important rule, that rustc will impose on us:

Which means that there can be zero or more immutable references "alive" at the same time, but there can only be zero immutable references if there is one (and only one!) mutable reference. This guarantees that no one reads a value, while it is being mutated. While this is useful for concurrent code, we’ll see in a bit that even single threaded code benefits from that guarantee.

But before we look into how this helps us eliminate an entire category of bugs from our code at compile time, rather than a random RuntimeException thrown at us in production, let us first look into how references and borrowing comes into play with struct and their impl blocks. First, how is a struct declared in Rust:

A struct in Rust is the equivalent of a class in Java. You first declare it using the struct keyword, followed by the name you want to give it. The fields it declares follow the Rust syntax we’ve seen so far: <name>: <type>. In the example above, the line declaring the person field is ended by a comma (,). This isn’t strictly necessary as it is the last (well the only) field in our Greeter struct. But fields that’d be declared before it would need the comma. It is considered good practice to always end field declaration with comma, so should you ever add a new one, the diff in your version control system would be cleaner.

For now our struct looks more like a C struct than a Java class. At this stage you probably expect a constructor and some methods. So let’s start with the former. In this particular case, you wouldn’t need a constructor per se, as since we put all this within the same file, visibility doesn’t matter. Which, as a matter of fact, is implicit in our examples, as the struct isn’t explicitly declared as pub. Neither is the field person. Which means that at any point in our main function, we could instantiate such as Greeter with the following syntax:

And, well, there is no constructor as you know them from Java in Rust neither. You could define a function on our struct Greeter that performs some argument manipulation for instance. Adding a function to a struct is done with an impl block for that same name. Below is an example of a Greeter::new function that creates a Person and returns it (the struct itself, not a reference to it):

Again, the function would be declared public with pub fn new, if we needed it exposed outside of this scope. We can see that the function takes ownership of a String, which means the caller would move the String ownership to this function. And that the function returns, denoted with the arrow → a… Self?

Self is a placeholder for the type of the impl block. This is useful when refactoring, or when the type is somewhat verbose. But you could just as well have the method signature be fn new(person: String) → Greeter. Follows the block, within curly braces, that is the function’s body. When creating the Greeter this time though, we are not explicitly telling that the field person is to own the variable of the same name. Unlike when we previously did associate it with a freshly created String: person: String::from("Jane"). Them having the same name, is the reason we don’t have to prefix it with person:. It’s considered redundant and as such not necessary. Finally, the single line of the body again misses a ; at the end, as we are returning the freshly created Greeter instance to the caller, i.e. moving ownership of it to the caller. It is also worth mentioning that this freshly created struct was created on the stack. Even if the bytes of the person 's name are eventually on the heap, as this is a String.

So Greeter::new is again the equivalent of a static method in Java. How would you add an instance method then? First, let consider what an instance method even is. An instance method is one that acts on the instance of the object itself. What happens at runtime is that a reference to the instance is implicitly passed to a plain old function. Rust makes this a little more explicit. Below is the code that adds a .hi() method to the Greeter struct:

A reference to… self? Lowercase self this time? Yes. That’s how Rust defines an instance method. The first argument being &self, which actually is a shorthand for self: &Self, so a binding named self that is a reference to a type of Self, i.e. Greeter. That first argument can either be a &self, a &mut self or a self. We’ve seen the self variant with our Option.unwrap() at the beginning, it’s an instance method that consumes the instance. So once invoked the binding used to invoke the method becomes useless. &mut self tells the compiler that we expect a mutable reference, because we’ll be mutating state related to this struct. One example would be a setter in Java, or e.g. a set_person method on our struct Greeter. In order to reassign a new provided String to our person field, the reference to would need to be mutable:

If we’d declare it with &self, the compiler would complain as we’re trying to mutate self when assigning the new value to the person field.

Below is our entire main.rs as it stands now with all these changes… and a few little ones more:

We now also have a getter to the person field, that returns a reference as well as a bye() method. That method’s body is slightly different than from the hi() one that we saw before. They both do effectively the same: get a reference of the person to print their name. On line 23, in the hi() method, we get the self.person field and invoke the .as_str() which returns a &str for us to use, while on line 28 , within the println! macro invocation, we grab a reference & to the self.person, i.e. a &String. In both cases we do not move the self.person out of our struct. That move would be illegal, we only reference the field.

Which, when we run it, now yields this result:

If we now use this struct to play a bit with borrows, we can test the rule of reference from before:

For instance what do you think would this do?

Well, it would not compile, as it violates our simple rule:

The issue becomes visible when you look at the method declaration of set_person(&mut self, person: Person). It takes a &mut self, so when invoking the method, we implicitly try to borrow greeter mutably. Which we cannot do because a immutable reference is still alive. If we would use greeter instead to invoke .hi() on, the immutable reference would be "dead" and the code would work…

What do you think would this other simple example does:

This one might be a little less obvious, as we are borrowing a reference to the person field of the greeter. But, thinking about it, you’ll probably not be surprised that the result if pretty much the same as before:

The "transitivity" of the borrow of .person through greeter itself is tracked by the compiler and as such we get the same error back. We can’t have set_person replace the String within greeter with a new one, while we still have a reference to it. Replacing it would imply Drop -ping the old value while it is still being in use. Remember there is no garbage collector to only clean it up after the last reference is outlived.

You might want to use this very simple example to experiment with the lifecycle of references and see how the rule of references plays out for you. As a Java developer it does take some time to get used to this rule and think more of how your data is handled by your code. Rust requires a little more thinking as where and how to store data and references.

If now you’ve been wondering why this rule is there or might have felt that at times it is more constraining than actually helpful. Other than the reference to greeter.person being modified concurrently by another thread, there isn’t much added value to this guard than the freeing of the value… or is there? Let’s consider this simple Java code:

We’re iterating over a list, printing out the values and removing them as we go. An this keeps on happening in "real" code, unlike this idiotic example. If you’ve been doing Java for a while, this one is probably obvious to you already, but here is the output of running this:

Well, we didn’t get too far on this one! A ConcurrentModificationException, on a single thread! Now let’s try the same code in Rust:

Let’s first explain what is going on here. We’re creating a Vec<i32> from the handy vec! a function-like macro again. The type of the elements in the vector is i32, signed 32 bit integer - Java’s int, as this is what integer literal evaluate to in Rust.

We then iterate over the reference to our vec?! Well… no. The for loop there has some syntactic sugar added by the compiler. for i in items - without the ampersand - would be the equivalent of for i in items.into_iter(). Where into_iter() moves the ownership of the Vec from vec to the iterator. Eventually each item’s ownership is moved into the loop. While i in &items is the equivalent of i in items.iter(), which will provide us with references to each element of vec in our loop and leave the collection items untouched.

The loop block should be straightforward by now. So what happens if we run this?

Well, the rule of references will make it impossible for us to grab a mutable reference to our items, which we implicitly do by calling .remove which unsurprisingly takes a &mut self to the Vec to mutate it. All that while we are using an immutable reference to iterate over the elements within that Vec.

What about this code tho:

This does not violate our rule of references. While there is a keyword mut present, it is not on a reference, but on the variable which owns the Vec<i32>. We then go on to nest iterations, which results in at most two immutable references alive at the same time, which is not against our rule. If anything, this code will have the compiler produce warnings if, after these loops, we don’t mutate our items as then: variable does not need to be mutable and the first i isn’t used: if this is intentional, prefix it with an underscore.

When an instance goes out of scope, the compiler will insert some code for us that is responsible for Drop -ping the instance. That means freeing up all resources it might hold on to: more instances, file descriptors, sockets, … and eventually free up the memory it used, like the memory on heap for our String instances.

Let’s start a little experiment in trying to understand how Rust deals with freeing up resources. We’ll build on this hi/bye "idiom" we’ve built with the Greeter, but change it slightly. We want it to only say "hi!" once and make sure it always says "bye". Let’s jump right into it:

Ok, so what do we know already? We’re declaring two struct types: Scope and TrackedScope. They both have the same structure, containing a heap allocated String field called name.

The Scope field has one function and one method. The function is straight forward, but adds some nice usability to our previous Greeter, it takes a &str and converts it to the heap allocated String before creating the struct Scope for us. The method start is slightly more interesting though. It doesn’t take reference &self but consumes self instead! But before doing so, it prints that the scope is now started and then returns a TrackedScope moving the String instance from the Scope instance into the newly created TrackedScope one that is then returned. So the name is preserved, but the original Scope instance is dropped.

Talking about Drop! We also have an impl block for TrackedScope, but it implements the Drop trait. The trait, your Java interface equivalent, declares a single method fn drop(&mut self), which we implement here by printing that the scope has ended. That drop method invocation is what the compiler will add for us automatically in the right spot. You cannot invoke that method manually in your code yourself though, the compiler (again!) will prohibit that.

If you manually want to drop something, and as such invoke the .drop() on the instance, there is a handy utility function available in the standard library for you to use: std::mem::drop. You might be wondering how special that function is and what kind of magic it uses to drop the actual instance that you couldn’t possibly be trusted with?! Well, let’s take a look! Below it the full source code of this mysterious utility function:

Wait?! What? It’s… it’s empty!

Yes it is. The magic happens in the function signature again, it take ownership of the argument passed in! The method is generic over T, without any bounds, so it’d accept anything. And do absolutely nothing with it! But since, when invoking that function, the ownership of _x is moved to this scope and then… nothing, well… it gets dropped! The compiler sees it goes out of scope and insert the necessary cleanup code for you!

So let’s use our Scope now to see how the drop -ing occurs in code, considering these few cases below:

Worth while to note is that we add a synthetic block on line 7, which ends on line 10. Also, the scope two is not bound to a variable when started. So what do you think the output should be? Let’s run this and see:

Probably unsurprisingly, three get dropped at the end of our block on line 10, right before four gets started. two though gets dropped right after it is started, as we never bind the returned TrackedScope to anything, so it can be cleaned up right away. While three only behaves similarly as it’s the last statement in that block. In the outer scope, we first started one then four, binding the result to a local variable and we can see that they get drop -ped in inverted order, first four, then one. Which makes sense. As a structure binding declared later could hold a reference to something declared before, hence dropping in reverse order cleans things up properly!

That was quite some content again. We’ve by now explored the concept of ownership much deeper. How there is only one owner of any given piece of data. But that ownership can be transferred, otherwise known as move. We’ve then looked at how we can borrow a reference to data, according to the rule of references. And finally saw how the rust compiler frees resources by Drop -ping our struct instances.

We’ve indeed done most of this on struct by declaring our own new types and adding functions and methods to them. By doing this, we’ve seen how three different types of methods that can be declared on structs: &self, &mut self and self and how these differ in their behavior with regards to the borrow checker and move semantics.

Finally, we got a glimpse at how a for loop works in Rust and it differs from Java’s. We’ve also illustrated some of the usages of the above principles with code from the Rust standard library. Just like with Java, that source code is available for you to read and it is strongly encouraged that you spend some time reading some of it to get further acquainted with how the Rust standard library developer write code to leveraging the language solve some basic problems, like we did for std::mem::drop().

I can’t possibly end this, without reiterating one advice: run cargo clippy often on your code! Spend the time refactoring what you’ve got once it compiled. And re-run clippy! Spending some time exploring the language on your own terms is probably the best way to get a better handle on writing Rust! Coming from Java, chances are that becoming comfortable with where data should reside, i.e. which parts of your program own what data, and own to then efficiently share that data might be somewhat challenging initially. But I do believe that finding the path of least resistance to how data flows through your code will eventually lead to you writing better Java too.

There is much more for us to cover. So expect some new post some time this month with the next adventure into Rust coming from Java! I hope this was insightful and, again, please don’t hesitate to leave comments, remarks, suggestions, fixes, complains…​