EzDevInfo.com

Dynamically create parboiled2 rules

Can I generate rules dynamically in parboiled2 parser? The use case is that I have a bunch of rules already defined, but want to add more and not compile every time I add a rule.

Understanding Parboiled2's '~' Combinator

Looking at the parboiled2 section, Rule Combinators and Modifiers:

I don't understand the a, b, and then a ~ b diagram.

So far I've found the documentation straightforward. But I am a bit lost here.

Can you please explain each block?

Parboiled2 Parser Example

I'm trying to try out this example from parboiled2:

scala> class MyParser(val input: org.parboiled2.ParserInput) 
            extends org.parboiled2.Parser { 
                def f = rule { capture("foo" ~ push(42)) 
                } 
        }
defined class MyParser

Then, I create a new MyParser with input of "foo".

scala> new MyParser("foo").f
res11: org.parboiled2.Rule[shapeless.HNil,shapeless.::
            [Int,shapeless.::[String,shapeless.HNil]]] = null

Yet the return value is null.

How can I run this simple f Rule from the REPL?

parboiled2 Illegal rule composition

I am writing an cron parser, but compiler complains illegal rule composition,

What's wrong with my parser?

import org.parboiled2._

sealed trait Part
case class Fixed(points: Seq[Int]) extends Part
case class Range(start: Int, end: Int) extends Part
case class Every(start: Int, interval: Int) extends Part
case object Full extends Part
case object Ignore extends Part

class CronParser(val input: ParserInput) extends Parser {

  def number = rule { capture(digits) ~> (_.toInt) }

  def digits = rule { oneOrMore(CharPredicate.Digit) }

  def fixed = rule { oneOrMore(number).separatedBy(",") ~> Fixed }

  def range = rule { digits ~ '-' ~ digits ~> Range }

  def every= rule { digits ~ '/' ~ digits ~> Every }

  def full= rule { '*' ~ push(Full) }

  def ignore = rule { '?' ~ push(Ignore) }

  def part = rule { fixed | range | every | full | ignore }

  def expr = rule { part ~ part ~ part ~ part ~ part}
}

Parboiled2 grammar for parsing escaped CSV line

I am trying to parse a single line which contains strings separated by delimiters into a sequence of these strings. It should be able to have any character in the strings, if a field contains a delimiter it needs double quotes around it. In order to have double quotes in such a field, the double quotes are escaped by .

I used this as a starting point: https://github.com/sirthias/parboiled2/blob/695ee6603359cfcb97734edf6dd1d27383c48727/examples/src/main/scala/org/parboiled2/examples/CsvParser.scala

My grammar looks like this:

class CsvParser(val input: ParserInput, val delimiter: String = ",") extends Parser {
  def line: Rule1[Seq[String]] = rule {record ~ EOI}
  def record = rule(oneOrMore(field).separatedBy(delimiter))

  def QUOTE = "\""
  def ESCAPED_QUOTE = "\\\""
  def DELIMITER_QUOTE = delimiter+"\""
  def WS = " \t".replace(delimiter, "")

  def field = rule{whiteSpace ~ ((QUOTE ~ escapedField ~ QUOTE) | unquotedField) ~ whiteSpace}
  def escapedField = rule { capture(zeroOrMore(noneOf(QUOTE) | ESCAPED_QUOTE)) ~> (_.replace(ESCAPED_QUOTE, QUOTE))  } 
  def unquotedField = rule { capture(zeroOrMore(noneOf(DELIMITER_QUOTE))) }
  def whiteSpace = rule(zeroOrMore(anyOf(WS)))
}

When I call it with "quote\"key",1,2 I get Invalid input 'k', expected whiteSpace, ',' or 'EOI' (line 1, column 9)

What am I doing wrong? How would I debug this? (And as a bonus question: How would I extend the grammar to allow the delimiter to be multiple chars like ##?)

Thank you!

Parboiled2 PopRule example

After reading the documentation on https://github.com/sirthias/parboiled2, I discovered that I could pop something out of the stack with the rule:

type PopRule[-L <: HList] = Rule[L, HNil]

But I could not find a working example of this type of rule for when L is not String.

For example, supose I have the following rule:

case class A()
case class B()
def foo = rule { push(A) }
def pop_rule:PopRule[A, HNil] = rule { pop(A)}

To justify this there is the general definition of a parboiled2 rule:

class Rule[-I <: HList, +O <: HList]

Where it represents a rule that pops the value from I from the stack and puts the value from O into the stack.

So far, I cannot think of an example implementation for the following rule type:

def rule_of_interest:Rule[A, B] = rule { pops(A) ~> push(B)}

Pure Functional Programming with Path Dependent Types (Parsers) in Scala?

So when using Scala Parsers one may have:

case class MyParser(foos: List[String]) extends Parsers {
  val bars: List[Parser[String]] = foos.map(parserFromString)

  // Expensive function
  def parserFromString(s: String): Parser[String] = ???
}

Parser is a path-dependent type, so this code cannot be refactored so that bars can be passed in from the constructor. Note that parserFromString in my use case actually creates a MyParser in such a way that MyParser construction becomes O(N!) where N = foos.size.

So suppose now I wish to add to bars via another foo. The FP way would be to refactor to include bars in the constructor, then define something like def +(foo: String): MyParser = copy(bars = bars :+ parserFromString(foo), but as mentioned I can't I have to recreate from scratch.

My solution is just make bars a private var and mutate it with a Unit method update, i.e. def update(foo: String): Unit = bars +:= parserFromString(foo)

My first question is quite simple: am I stuck? Do I have to use mutation?

Second question: does Parboiled2 suffer from this? Do they use path-dependent types (at a glance it doesn't look like it), and if so why do Scala parsers use path-dependent types?

If parboiled2 does not suffer from path-dependent types this alone can be a reason to use it!

If anyone is wondering why I need to create Parsers from parameters it's because I'm implementing a language where users can define macros and use those macros in defining other macros. So no, before anyone tries telling me to change the design, I can't. I also really don't want mutability since I'm going to be needing multi-threading later.

Parboiled2: reference to position in source text from AST

I am writing a DSL, and learning parboiled2, at the same time. Once my AST is built, I would like to run some semantic checks and, if there are any errors, output error messages that reference the offending positions in the source text.

I am writing things like the following, which, so far, do work:

case class CtxElem[A](start:Int, end:Int, elem:A)

def Identifier = rule {
  push(cursor) ~
  capture(Alpha ~ zeroOrMore(AlphaNum)) ~
  push(cursor) ~
  WhiteSpace
  ~> ((start, identifier, finish) => CtxElem(start, finish, identifier))
}

Is there a better or simpler way?