The Magic Mixin, Simple Version

For boilerplate equality of classes, where the performance of equals is not all that important. I use a simple trait that I implement a few methods from. It's pretty handy so I figured I would post it.

Check out the Gist

Updates on the Intellij Plugin, Error Console is Less Grabby

It's still a work in progress but I have managed to make the errors window less grabby. Right now it prints some harmless messages to standard out, just ignore them.

As before:

If you are an Intellij user and want to check out the modified plugin, here is the source and here is a download of a jar that you can drop in your /plugins folder

Intellij Plugin

I recently switched to using Intellij 8 for scala development from Eclipse. On the whole it is a lot more polished than the current incarnation of the Scala IDE for Eclipse, I recommend it highly. Out of the 3 big java IDEs Intellij has the best Scala support at the moment. I will try out the Scala IDE for Eclipse again, once Miles gets done with all the refactoring he is doing.

One of the features I missed most from Eclipse was how eclipse compiles automatically in the background when files are saved. Luckily there is a plugin for Intellij (mentioned in this faq) called EclipseMode to add that behaviour, though it has a few kinks. [1]

I wasn't able to fix all of the issues I have with it (yet), however I did make one simple change that adds a toggle for it (called Compile On Save) to the build menu.

If you are an Intellij user and want to check out the modified plugin, here is the source and here is a download of a jar that you can drop in your /plugins folder

[1] kinks
One caveat, as others have mentioned: Is there any way to keep the messages window from auto-opening and from stealing focus? I tried mucking around with the plugin source, and couldn't figure it out, seems that problems view is not part of the openAPI and very hard to get to.

Also it doesn't seem to work with intellij 9, (tho i was able to hack it to get it to work, seems like they changed some of the non open api stuff the plugin uses).

GetAnyClass Tip

I've been having some trouble with calling getClass on generic/primitive types.
It seems that it's a known issue with scala that wontfix and hence this code won't compile

def convert[A,B](instance : A, toClass : Class[B]) : Option[B] = {
//Fails here with the error below
instance.getClass
...Rest of implementation...
}

(The error):

:6: error: type mismatch;
found : A
required: ?{val getClass: ?}
Note that implicit conversions are not applicable because they are ambiguous:
both method any2stringadd in object Predef of type (Any)scala.runtime.StringAdd
and method any2ArrowAssoc in object Predef of type [A](A)ArrowAssoc[A]
are possible conversion functions from A to ?{val getClass: ?}
instance.getClass
^

the problem has a quick fix though suggested by paulp on the #scala irc channel on Freenode:

you can encapsulate that though:
scala> def getAnyClass(x: Any) = x.asInstanceOf[AnyRef].getClass
getAnyClass: (x: Any)java.lang.Class[_ <: java.lang.Object
casting to AnyRef (for this purpose) is a no-op on non-primitives, so...
you could plant that as an implicit if you want so 1.getAnyClass will work.

what this translates to is basically defining a class like this somewhere in your project:

/** Predef for brewmaster, maybe package objects will obsolete this
*/
private[brewmaster] object Brewdef {
implicit def any2GetAnyClass(x : Any) = new GetAnyClass(x)
class GetAnyClass(val x : Any) {
def getAnyClass = x.asInstanceOf[AnyRef].getClass
}
}

then chaning the code above slightly to use the .getAnyClass method it with

Publish Post

import Brewdef._
def convert[A,B](instance : A, toClass : Class[B]) : Option[B] = {
instance.getAnyClass
...
}

voila!
this will also allow things like 5.getAnyClass to compile (.getClass fails)

Immutable Objects and Named Parameters in scala

--Updated--
Jorge Ortiz pointed out that 2.8 is already going to include a copy method in every case class:

Every case class already comes with a "copy" method defined something like:

def copy(x: Int = this.x, y: Int = this.y, z: Int = this.z) = {
Point3d(x, y, z)
}

And used like:

p1.copy(z = 2)

-Jorge Ortiz

Here's a cool way the named parameters feature being added in Scala 2.8 might help with immutable objects. It should help clients avoid errors when copying an immutable class, also it should end up hiding them from certain changes in that class.

//An overly simple example class
case class Point3d(x : Int, val y : Int, val z : Int) {
//Add method like this to make copying immutable classes easier
def copy (x : Option[Int] = None, val y : Option[Int] = None, val z : Option[Int] = None) = {
Point3d(x.getOrElse(this.x), y.getOrElse(this.y), z.getOrElse(this.z))
}
}

//Original class
val p1 = Point3d(1,1,1)

//"Old Way"
//notice every field is copied, this
//gets nasty as the class acquires more than just 3 fields
//Also notice how directly tied to the factory method
//this style is
val pOld = Point3d(p1.x, p1.y, 2)

//"New way"
//here we only care about the part of the
//class that we want to change. The class we are
//changing was refactored and additional fields were
//added to it, we would have to change all the "Old Way"
//calls, however this call will still work fine.
val pNew = p1.copy(z=Some(2))

Hashcode & Equals in the Scala 2.8 Collections Library

[Edit]
Martin commented that the disparity between equals and hashCode has been changed on the Scala 2.8 trunk so that they now follow the hashCode-equals contract, therefore most of this post is irrelevant.

I just finished reading the whitepaper on the collections library that is going to be put into scala 2.8

First let me say that I really enjoy the new collections library, it seems like it is going to be a vast improvement over the old, and give scala one of the nicest, most growable collections libraries of any language out there. I really think that the new library with how well abstracted everything is will be a really cool starting point for allowing users to grow the language and give Scala what will in a very short time could be the end-all be-all of collections libraries.

Having just watched Guy Steele's talk on Growing A Language (thanks to debasishg@twitter) for tweeting it. It seems like Martin Odersky and the other architects of the new library really channelled what he was getting at in his talk. This new library while being fairly complete in it's own right, is going to give users a both the flexibility to implement collections that are highly optimized and/or the ability to implement them very quickly by reusing existing code. It will let the vendor of the Collection class focus more time on his implementation and less time on having to provide a bazillion methods that clients reasonably expect from a collection class.

There was a one thing that jumped out at me as being a bad idea in the new library, it was on page 15 and relates to how the new collections library is going to handle hashCode for mutable collections, and equals for all collections.

Per the whitepaper:

It does not matter for the equality check whether a collection is mutable or immutable. For a mutable collection one simply considers its current elements at the time the equality test is performed. This means that a mutable collection might be equal to different collections at different times, depending
what elements are added or removed. This is a potential trap when using a mutable collection as a key in a hashmap.
Example:
val map = HashMap[Array[Int], Int]()
val xs = Array(1, 2, 3)
map += (xs -> 6)
xs(0) = 0
map(xs)
In this example, the access on the last line will most likely fail because the hashcode of the array xs has changed in the second-to-last line. Therefore, the hashcode-based lookup will look at a different place than the one where xs was stored.
To avoid this trap, mutable collection classes will not support taking their hashCode—they will throw an UnsupportedOperationException with the message “unsuitable as hash key” if hashCode is called

I have a very bad feeling about any class throwing an exception when you ask it for its hashCode, the same way I would have a bad feeling if a class throw an exception just for calling its toString method, or its equals methods. I am generally not a big fan of surprise exceptions and while I am well aware of the dangers of using mutable field in a hash map, I just don't think that the hashCode method should be throwing exceptions. It's going to be a real gotcha for any programmer.

Programmer add gotcha to brain: "I can generally call hashCode on any object, except for this sub-type of objects, which are going to throw an exception at me because it is dangerous to use them in hashmaps. If I am writing generic code that calls hashCode on object I need to catch exceptions, because, certain objects decide they really did not want to ever be used in a hashmap, even though my code doesn't mutate them afterwards, as I am just sending these objects right out of the system for serialization and I need a good key to use for them that obeys the contract of hashcode. In this case, I am going to be in trouble as there is now no longer a guaranteed way for me to get hashCode that obeys the contract of hashCodes/equals. What do I do?"

Further, If a mutable collection is excluded from being used as a key because it could be mutated after being added to a map, shouldn't any 'case class' with a var field also be Unhashable? (A Case Class in scala provides some syntatic sugar by auto-implementing the equals and hashCodes methods based on the fields of the class.) Isn't Unhashable really just a synonym for Mutable? And is the real issue here forcing all client's to never use a mutable value as a key in a hashmap? And shouldn't then any mutable class throw UnsupportedOperationException when the hashCode of that class is called? Is it really a good idea to enforce this constraint this hard?

Having said that, and given the subtlety of the errors that happen when programmers do accidentally use mutable values as hash keys, then change them later on, I think that making some sort of bell go off that says: "Warning, you just used a mutable value as a key in a hashmap! I hope you know what you're doing, because you probably just messed up" is a good idea.

Another thing that scares about the 'This is Unhashable because it is mutable ideology' is that it is going to force everyone to implement their own hash functions for objects that are mutable, or always first coerce them into an immutable version just to get a hashCode (with possible severe performance impacts), if they need to for some reason get a hashCode for that class (maybe for some sort of db serialization scenario, who knows). Since their is no integrated way to get a hashCode for a mutable collection, they now are going to have make their own hashing function, and that is generally a bad idea, as making good hash functions can be hard. It's better to do it right once in the library than to have everyone do it their own way. Can it be said with absolute certainty that just because something is mutable, it must never generate a hashCode for any reason?

It would be really nice if it could be enforced by the compiler as a constrait on the type parameters of HashMaps, namely the Key type of a hashmap should be able to somehow say something like:
HashMap[K <: !Unhashable, V]
where K <: !Unhashable means K is not a subtype of Unhashable, and the compiler enforces the constraint, rather than the runtime.


The other thing that stuck out was a potential gotcha around the way equality is handled.

The Whitepaper on Equality:

The collection libraries have a uniform approach to equality and hashing. The idea is, first, to divide collections into sets, maps, and sequences. Collections in different categories are always unequal. For instance, Set(1, 2, 3) is unequal to List(1, 2, 3) even though they contain the same elements. On the other hand, within the same category, collections are equal if and only if they have the same elements (for sequences: the same elements in the same oder). For example, List(1, 2, 3) == Array(1, 2, 3), and HashSet(1, 2) == Treeset(2, 1).

From above: seems that a List(1,2,3) will now equal an Array(1,2,3) while they have different hashCodes (List is some defined value, and Array is Nothing because it throws an exception) which violates the contract of hashCode. Maybe this doesn't matter that much in this case because the mutable collections can never be used in a hashmap, but what about two different implementations of an immutable collection within the same category? If this is the direction being taken, then clients probably should never should override equals or hashCode for their own implementations of collections. Conceptually equals and hashCode should be final at the level of Map, Set, and Sequence otherwise any client who writes their own version of a collection inheriting from one of these has to ensure that they generate the same hashCode for all other collections of the same category that contain the same values or else they risk breaking the contract put forth in the whitepaper.

Another approach might be to just make a method in Sequence, Map, & Set called sameElements (or deepEquals or equalElems or whatever) that returns true iff these two collections are of the same category and contain the same elements.

I am not really sure which is preferable, on the one hand I like the idea of treating all sets, maps, and sequences uniformly, in the sense that if they have the same values, then they are the same, on the other hand, It still would be nice to quickly tell if certain maps have the same ordering as well they are iterated across (I suppose that something like this could probably do that as well):
pseudocode
def sameOrder = (map1.iterator zip map2.iterator).find((e1, e2) => {e1 != e2}).isEmpty

Planning to do a scalide screencast sometime

I am hoping to learn how to record a brief 5 minute screen cast in the next few weeks to walk through some of the features of the interpreter. I know that the user interface leaves a lot to be desired and I haven't had much time to work on it, however hopefully I can explain it better for people new to Scala as it can be a good way to help learn the language. (I use it as a calculator now). To people who have used mathematica before the interface might be only extremely horrible, for everyone else it's probably entirely unusable ;-)

I am also going to look over the recent changes that have been checked into the scala tools package by extempore to see if I can add some auto-complete support which would make the interpreter much more usuable, stay tuned :-)