Examples and Internals of slick-codegen

2017年12月16日 6:00

だいたい 27 分で読めます

development

Table of Contents

Top
Background
slick-codegen Code
Overall Image
Sample Collection
Converting java.sql.time to org.joda.time.DateTime
Setting Default Values
Automatically Converting to JSON
How <code>xxxRow</code> Is Generated
Creating a <code>Def</code>
Using Custom ID Types
ID Type Generator
Rewriting Table ID Columns
Adding Mappers for ID Types
PathBindable Generator
Conclusion
Code Used

This is the 17th article of Scala Advent Calendar 2017.

Yesterday, @yoshiyoshifujii wrote about "Considering Errors from Infrastructure Layers".

Background

slick-codegen is a program that automatically generates code corresponding to a DB schema. It is a reliable tool that eliminates the need to write similar code repeatedly. If you are using Slick, you may have used it at least once.

Although slick-codegen is introduced in the official documentation (Schema Code Generation), the explanation is insufficient, making it difficult to understand how to customize it in practice. Additionally, there is a sample repository (slick/slick-codegen-customization-example) that introduces some usage methods, but there are many other ways to use it. Reading the API documentation is also helpful, but it would be nice to know what can be done more casually.

Therefore, I will introduce some samples and explain how to customize them.

slick-codegen Code

Sometimes, reading the code is the best way to understand it. slick-codegen consists of five small Scala codes, so it's not difficult to read.

Overall Image

To help you understand, I will briefly explain the overall structure.

A custom generator that you create inherits from SourceCodeGenerator.

class CustomGenerator(model: m.Model) extends SourceCodeGenerator(model) {
  ...
}

SourceCodeGenerator implements AbstractSourceCodeGenerator and OutputHelpers.

class SourceCodeGenerator(model: m.Model) extends AbstractSourceCodeGenerator(model) with OutputHelpers{
  ...
}

Furthermore, AbstractSourceCodeGenerator uses AbstractGenerator.

abstract class AbstractSourceCodeGenerator(model: m.Model) extends AbstractGenerator[String,String,String](model) with StringGeneratorHelpers{
  ...
}

Note: StringGeneratorHelpers is defined in the same file; it’s not central to the discussion, so we’ll omit it here.

AbstractGenerator uses GeneratorHelpers.

abstract class AbstractGenerator[Code,TermName,TypeName](model: m.Model) extends GeneratorHelpers[Code,TermName,TypeName]{ codegen =>
  ...
}

So, the relationship between classes is as follows:
Your class ← SourceCodeGenerator ← AbstractSourceCodeGenerator ← AbstractGenerator
Each class is defined in a file with the same name, so it's easy to follow the code.

Sample Collection

For the sample schema, I prepared the following SQL and executed it on MySQL.

create table user(
   id INT NOT NULL AUTO_INCREMENT,
   name VARCHAR(255) NOT NULL,
   created_at datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
   PRIMARY KEY ( id )
);

create table item(
   id INT NOT NULL AUTO_INCREMENT,
   name VARCHAR(255) NOT NULL,
   user_id INT,
   FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE,
   PRIMARY KEY ( id )
);

Converting java.sql.time to org.joda.time.DateTime

First, I will introduce a sample that converts java.sql.time to org.joda.time.DateTime. This is a convenient feature.

To start with, the first sample is from a reference:

In that example, the following customizations are demonstrated:

Add required imports
Make auto-increment values an Option (Table.autoIncLastAsOption; in later samples I use Table.autoIncLast plus Column.asOption as the former is deprecated)
Convert column raw types (Column.rawType)
Exclude specific columns by name by transforming the model

Setting Default Values

Next, I will explain how to set default values. This is useful when you want to set default values for columns like createdAt and updatedAt. We can make constructors convenient by assigning defaults:

case class UserRow(id: Int, name: String, createdAt: org.joda.time.DateTime = DateTime.now)

This allows creation with only the required parameters: UserRow(1, "name").

In slick-codegen, override AbstractGenerator.ColumnDef.default to emit defaults. Return None when absent, or Some(value) when present:

override def default = rawName match {
  case "createdAt" | "updatedAt" => Some("DateTime.now")
  case _ => super.default
}

Automatically Converting to JSON

Then, I will introduce a sample that automatically converts to JSON. This is useful when you want to return JSON data in an API. In tests or simple APIs, you may want to return an xxxRow as JSON directly:

def someAction = Action { ...
  val user: UserRow = ...
  Ok(Json.toJson(user))
}

To enable this, define implicit Writes in the companion object:

object UserRow {
  implicit val jsonWrites = Json.writes[UserRow]
}

Generating one by one is tedious, so we'll auto-generate these companions as well. Note: once you generate a Row companion, UserRow.tupled becomes a compile error; switch to (UserRow.apply _).tupled to fix it.

How `xxxRow` Is Generated

To append a companion object under xxxRow, we first need to know what xxxRow actually is.

For each table, Slick creates a TableDef. The finished code is gathered via TableDef.code.

The xxxRow itself is an Entity backed by EntityTypeDef.

Those entity/type definitions are collected under TableDef.definitions and turned into code by TableDef.code.

def definitions = Seq[Def]( EntityType, PlainSqlMapper, TableClass, TableValue )
def code: Seq[Code] = definitions.flatMap(_.getEnabled).map(_.docWithCode)

Creating a `Def`

Based on the above, we can create our own Def and insert it in definitions to emit a companion object next to each xxxRow. Since the class name matches the Entity, we can reuse EntityTypeDef and override only what we need—in this case, just the code.

class EntityCompanionDef extends EntityTypeDef {
  override def doc = ""
  override def code =
    s"""object ${rawName} {
       |  import play.api.libs.json._
       |  import play.api.libs.json.JodaWrites
       |  import play.api.libs.json.JodaReads
       |
       |  val dateFormat = "yyyy-MM-dd'T'HH:mm:ss.SSSZ"
       |
       |  implicit val dateTimeWriter = JodaWrites.jodaDateWrites(dateFormat)
       |  implicit val dateTimeJsReader = JodaReads.jodaDateReads(dateFormat)
       |  implicit val jsonWrites = Json.writes[${rawName}]
       |  implicit val jsonReads = Json.reads[${rawName}]
       |}
     """.stripMargin
}

override def definitions = {
  val companion = new EntityCompanionDef
  Seq[Def](EntityType, companion, PlainSqlMapper, TableClass, TableValue)
}

This also demonstrates JodaTime string conversions. The doc can be the empty string.

Running this yields definitions with the companion objects appended for each Entity.

However, as mentioned earlier, UserRow.tupled fails to compile when a companion is added. Apply a replacement in TableDef.code:

override def code = {
  super.code.toList.map(_.replaceAll(s"${EntityType.name}.tupled", s"(${EntityType.name}.apply _).tupled"))
}

Using Custom ID Types

Finally, I will explain how to use custom ID types. This is useful when you want to distinguish ID types for each resource.

Entities’ id types change from Option[Int] to Option[UserId] and so on:

case class UserRow(name: String, createdAt: DateTime = DateTime.now, id: Option[UserId] = None)

We will follow the approach in this article and also generate PathBindable:

ID Type Generator

For clarity, place ID types in a separate file from table definitions. We want to start at the class definitions, so the auto-generated header gets in the way.

A typical header looks like this:

// AUTO-GENERATED Slick data model
/** Stand-alone Slick data model for immediate use */
object IDs extends {
  val profile = slick.driver.MySQLDriver
} with IDs

/** Slick data model trait for extension, choice of backend or usage in the cake pattern. (Make sure to initialize this late.) */
trait IDs {
  val profile: slick.jdbc.JdbcProfile
  import profile.api._
  import slick.model.ForeignKeyAction
  import slick.jdbc.{GetResult => GR}

  /** DDL for all tables. Call .create to execute. */
  lazy val schema: profile.SchemaDescription = Item.schema ++ User.schema
  @deprecated("Use .schema instead of .ddl", "3.0")
  def ddl = schema

  // ...
}

Remove this by overriding the generator:

override def code = tables.map(_.code.mkString("\n")).mkString("\n\n")
override def packageCode(profile: String, pkg: String, container: String, parentType: Option[String]) : String =
  s"""package models
     |
     |${code}
   """.stripMargin

You will then get output like:

package models

class ItemId private (private[models] val value: Int) extends AnyVal {
  override def toString = value.toString
}
object ItemId {
  // ...
}

class UserId private (private[models] val value: Int) extends AnyVal {
  override def toString = value.toString
}
object UserId {
  // ...
}

Rewriting Table ID Columns

(This depends heavily on your schema; here we assume primary key column is id, table names are singular, and foreign keys are named (table)_id.)

Overriding the primary key column type is easy, but foreign keys need extra handling:

override def Column = new Column(_) {
  override def rawType = model.tpe match {
    case _ if model.name == "id" => s"""${TableValue.name}Id"""
    case _ => super.rawType
  }
}

Collect foreign-key column names from the model and pass them into your generator:

val idColumnNameSet = (for { t <- model.tables } yield s"${t.name.table}_id").toSet

This builds a set like:

Set(item_id, user_id)

Pass the set into your custom generator:

class CustomTableGenerator(model: m.Model, idColumnNameSet: Set[String]) extends SourceCodeGenerator(model)

Use that to switch types:

override def rawType = model.tpe match {
  case _ if idColumnNameSet.contains(model.name) => model.name.toCamelCase
  case _ if model.name == "id" => s"""${TableValue.name}Id"""
  case _ => super.rawType
}

Adding Mappers for ID Types

Provide implicit mappers so Slick can map your ID types:

implicit val userIdMapper = MappedColumnType.base[UserId, Int](_.value, UserId.apply)

Or generate them from names:

def implicitIdMapper(name: String): String = {
  val idName = s"${name.toCamelCase}"
  val uncapitalizedIdName = idName.head.toLower + idName.tail
  s"implicit val ${uncapitalizedIdName}Mapper = MappedColumnType.base[${idName}, Int](_.value, ${idName}.apply)"
}

This is easy to generate using the Set we prepared earlier.

implicit val itemIdMapper = MappedColumnType.base[ItemId, Int](_.value, ItemId.apply)

PathBindable Generator

Finally, define PathBindable so each ID works in conf/routes. This can live in a separate file; see the combined code at the end.

It’s almost the same as the ID type generator, so refer to the final combined code.

object PathBindableImplicits {
  import play.api.mvc.PathBindable
  implicit val userIdBindable: PathBindable[UserId] = PathBindable.bindableInt.transform(UserId.apply, _.value)
  // ...
}

Conclusion

Through these samples, I walked through slick-codegen internals and a few deeper customization techniques.

Because the output is easy to customize, the same approach can be applied outside of Slick as well. Lean on slick-codegen and automate as much boilerplate as possible.

Tomorrow’s Scala Advent Calendar 2017 entry is an introduction to sangria by @grimrose@github. I’m planning to jump into GraphQL myself—looking forward to the article!

Code Used

Below is a tiny driver that wires the generators together (abbreviated):

object Generator extends App {
  val slickDriver = "slick.jdbc.MySQLProfile"
  val outputFolder = "./app/models"
  val pkg = "models"

  // Load model (omitted)
  val model: m.Model = ???

  val idColumnNameSet = (for { t <- model.tables } yield s"${t.name.table}_id").toSet
  new CustomTableGenerator(model, idColumnNameSet).writeToFile(slickDriver, outputFolder, pkg, "Tables", "Tables.scala")
  new CustomIDGenerator(model).writeToFile(slickDriver, outputFolder, pkg, "IDs", "IDs.scala")
  new PathBindableGenerator(model).writeToFile(slickDriver, outputFolder, pkg, "PathBindableImplicits", "PathBindableImplicits.scala")
}