Marcin Kossakowski

Marcin Kossakowski

Tech and stuff

24 Dec 2018

Scala Extension Methods Using Implicits

What are extension methods?

Implicits are one of the fundamental language features in Scala. Some of its uses are in implicit method parameters, implicit object conversions and described in this article – implicit extension methods. This methodology is also called “pimp my library pattern”.

Extension methods allow you to add new capabilities to existing types / classes without the need of extending original type. This is especially useful when we want to enrich some class we do not have control over (e.g. external library).

A good example of this is in Apache Spark when adding support for new data sources for read write, e.g. Avro IO support. All we need to do is to add avro dependency to our project, add an import in the scope and magically our Dataframe/Dataset API has a new method available, e.g:

val df = spark.load.avro(...)

You can see implementation of this here

Use case example

In order to demonstrate this functionality let’s add new capablity to String type providing additional methods to working with file paths.

What we would like to have:

  • build path by composing strings with path delimiter character instead of glueing strings with (+) operator
  • ability to expand path to user home directory
  • easily convert path into File object

Here’s the complete code that implements above requirements.

import scala.language.implicitConversions
import java.io.File

object StringExtraImplicits {
  implicit class ImprovedString(s: String) {
  val sep = java.io.File.separatorChar

  def /(z: String): String = {
    z match {
      case z if s == "~" => System.getProperty("user.home") + sep + z
      case _ => s + sep + z
    }
  }

  def /(z: File): File = {
    new File(s + sep + z.toString)
  }

  def asFile = new File(s)
  }
}

And you can use it like this:

import StringExtraImplicits._

// Absolute path --> /tmp/foo.txt
"" / "tmp" / "foo.txt"

// Expanded user home path --> /home/marcin/Downloads/foo.txt
"~" / "Downloads" / "foo.txt"

// Relative path  --> src/main/resources/foo.txt
"src" / "main" / "resources" / "foo.txt"

// Convert to instance of File
val f: File = "" / "tmp" / "foo.txt".asFile

How does it work?

Implicit conversions has to be enabled either by importing scala.language.implicitConversions or using compiler flag -language:implicitConversions It is required to define implicit classes in an Object. To make it clear to the user it’s a good idea to include “Implicits” in the object name.

When Scala compiler encounters a call to a method that doesn’t exist on the object, it will look into surrounding scope for implicit classes that can provide such capability. When implicit class is found Scala will use the instance of that class to call extension method on it making it seem like method is now member of String type.

In above example I added overloaded (/) method that takes File type. This is to make call to “asFile” a little bit more friendly. Without it we would have to enclose our path “expression” in parentheses e.g.

("" / "tmp" / "foo.txt").asFile

// instead we can do: 

"" / "tmp" / "foo.txt".asFile

Also by enabling postfix operator feature we can call it without the dot!

import scala.language.postfixOps

"" / "tmp" / "foo.txt" asFile

Conclusion

Extension methods can be very useful and they are often used in Scala libraries but it is also very important not to overuse them as it may become very hard to understand the code.

Tags

scala