parsekit
Objective-C Tokenizer and Parser Generator. Supports Grammars.
Parse Kit -
I'm using ParseKit for objective-C which takes a BNF-like syntax for specifying grammers:
@start = command+;
command = new;
new = 'new' object ';';
object = 'house' | other;
Inclusion of the last line causes an error.
Basically I want to say an object can be a house or something else. The non-terminal element "other" is supposed to catch whatever word was there that wasn't house.
Am I going about the "anything-here" idea the wrong way?
Thanks!
Source: (StackOverflow)
Given a simple Parsekit grammar.
@start = sentence+;
sentence = 'beer' container;
container = 'bottle' | 'cup';
If I have a partial parse beer is it possible to get Parsekit to return the possible completions to the sentence?
Source: (StackOverflow)
I need to install ParseKit to compile with cocoa under Mac Os X, I use xcode 4.
I have searched online but there is only a guide for installing parse kit for iPhone.
Where do I find the download for Mac Os X and/or a guide?
Source: (StackOverflow)
Problem
I need to understand how TextKit works and how I can use it to build a text editor. I need to figure out how to draw ONLY the visible text the end-user interacts with or determine how I would go about lazily applying attributes to the visible text only without applying attributes to the entire changed range of text in the processEditing method.
Background
iOS 7 came out with TextKit. I have a tokenizer and code that fully implements TextKit (refer to Apple's TextKitDemo project -- a link is provided below)... and it works. However, it is REALLY slow. As text is parsed, by the NSTextStorage, it requires you to colorize the ENTIRE range of the edited text, in the processEditing method, on the same thread. Offloading the work to a thread doesn't help. It's simply too slow. I got to the point where I can re-attribute only the modified ranges, but if the range is too big the process will be slow. In some conditions the entire document could be invalidated after a change has been made.
Here are some ideas I have. Please let me know if any of these will work or maybe nudge me in the right direction.
1) Multiple NSTextContainers
Reading the docs it appears that I can add multiple NSTextContainers within an NSLayoutManager. I'm assuming that by doing this I should be able to define not only the number of lines that can be drawn in the NSTextContainer, but I should also be able to know which NSTextContainer is visible to the end-user. I know that if I go this route I will need to invest a LOT of time just to see if it's feasible. Initial testing suggests that you only need one NSTextContainer. So I would have to subclass NSLayout or create a wrapper where the layout manager determines which text goes into which text container. Yuck. Also, I have NO idea how TextKit lets me know that it is time to draw a particular NSTextContainer... maybe this isn't how it works!
2) Invalidating Ranges w/ NSLayoutManager
Invalidating the layoutManager using the invalidateLayoutForCharacterRange:actualCharacterRange:. But what does this actually do and how will it offload the text attribution phase? When does it let me know that a particular text needs to be highlighted? Also, I see that the NSLayoutManager will lazily draw glyphs... how? when? How does this help me? How do I tap into this call so that I can attribute the backing string before it actually lays out the text?
3) Over-riding the NSLayoutManager drawGlyphsForGlyphRange:atPoint: method.
I really do not want to do this. That being said, in Mac OS X, NSAttributedStrings have this concept of temporary attributes where the style information is used only for presentation. This speeds up the process of highlighting GREATLY! The problem is, it doesn't exist in the iOS 7 TextKit framework (or it's there and I just don't know about it). I believe that by over-riding this method it will give me the same type of speeds you would get by using temporary attributes... as I could answer all of the layout, color and formatting questions in this method without ever touching the NSTextStorage attributed string. The only problem is, I don't know how this method works in relation to other methods provided in the NSLayoutManager class. Does it keep state of the width and height? Does it modify the size of the NSTextContainer when it's too small? Also, it only draws glyphs for characters that have been added in the text buffer. It doesn't re-draw the whole screen. only a tiny section of it... and that's perfectly fine. I have some ideas of how I could work with this... but I really have no desire to layout the glyphs. That is WAY too much work and I haven't found a good example that does this.
I would greatly appreciate any help you have to offer.
As a thank you, I'm listing all of the frameworks and references I have used over the last few years that have helped me get to where I am now in the hopes that they are helpful to you.
Syntax highlighting frameworks:
Resources:
Most of these frameworks are the same. They either do not account for context switching (or you have to write the wrapper to provide context) for ranges or they do not repair context ranges as a user modifies the text (such as strings, multi-line comments, etc.). The last requirement is VERY important. Because if a tokenizer can't determine which ranges are affected by a change you will end up having to parse and attribute the entire string again. The only exception to this is the Crimson Editor. The issue with this tokenizer is that it doesn't save state at the time of tokenizing. At draw time, the algorithm uses the tokens to determine the state of drawing. It starts from the top of the document, up until it gets to the visible range of text. Needless to say, I have optimized this by cacheing the state of the document in certain parts of the document.
The other issue is that the frameworks do not follow the same MVC pattern that Apple does -- which is to be expected. The frameworks that have a complete working editor all use hooks, provided by the API they are built on (ie GTK, Windows, etc.), that provides them with information of where and when to draw to what part of the screen. In my case, TextKit appears to require you to attribute the entire changed range in processEditing.
Maybe my observations are wrong. (I hope they are!!) Maybe, ParseKit for instance, will work for what I need it to do and I simply don't understand how to use it. If so, please let me know! And thanks again!
Source: (StackOverflow)
I'm really stumped trying to get the ParseKit.framework (this) to work in general, not even bothering to implement it till it runs the demo app that comes with it.
What happens is the compiler can't locate < Foundation/Foundation.h> or something, which I thought the header was in the linked framework. Exact error: "Lexical or Preprocessor Issue: 'Foundation/Foundation.h' file not found."
Here's the code, just from the ParseKit_Prefix.pch:
//
// Prefix header for all source files of the 'ParseKit' target in the 'ParseKit' project.
//
#ifdef __OBJC__
#import <Foundation/Foundation.h>
#endif
Nothing unusual about it, did I mess up the file paths some how? I've reinstalled Xcode, re-downloaded the ParseKit, and nothing is helping. The suggestions here did nothing and it's not this. When I make a new project or use a different project and load the Foundation.framework and #import the header it works just fine. If I unlink the framework I can't find it to re-link again. Has anyone else had this kind of problem? Did I download it wrong somewhere? I have a very difficult time finding where exactly the Xcode UI links stuff, apple must get a kick out of frustrating people, so if anyone has anything they can think of please give me some feedback, I'm horribly confused right now. Thanks,
Source: (StackOverflow)
How should I use callback functions in parsekit? suppose I have the following rule:
expr_s = expr_p '+' expr_s | expr_p ;
should I pop 3 symbols from the resulting PKAssembly and add first and last numbers and then push the answer back to the stack?
And for the above rule how should I know it is the first or the second rule that caused a match?
I don't understand the order in which ParseKit calls callback functions. I could really use some help.
Thanks Todd for your response, keeping in mind your instructions I wrote the following grammar and callback functions for a simple mathematics expression that includes addition and multiplication:
- (IBAction)press_equals:(id)sender {
NSString *g = @"@start = expr_s; expr_s = expr_p ('+'! expr_p)+ ; expr_p = Number ('*'! Number)+ ;";
PKParser *p = [[PKParserFactory factory] parserFromGrammar:g assembler:self];
NSString *s = @"3*4+4*8";
[p parse:s];
PKAssembly *res = [p parse:s];
NSLog(@"res %@", res);
}
- (void)parser:(PKParser *)p didMatchExpr_s:(PKAssembly *)a {
NSLog(@"%s %@", __PRETTY_FUNCTION__, a);
NSArray *toks = [a objectsAbove:nil];
double total = 0.0;
for (PKToken *tok in toks) {
total += tok.floatValue;
}
a.target = [NSNumber numberWithDouble:total];
}
- (void)parser:(PKParser *)p didMatchExpr_p:(PKAssembly *)a {
NSLog(@"%s %@", __PRETTY_FUNCTION__, a);
NSArray *toks = [a objectsAbove:nil];
double total = 1.0;
for (PKToken *tok in toks) {
total *= tok.floatValue;
}
a.target = [NSNumber numberWithDouble:total];
}
and here is the output I get:
2012-04-06 22:54:31.975 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_p:] [3, 4]3/*/4^+/4/*/8
2012-04-06 22:54:31.976 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_p:] [4, 8]3/*/4/+/4/*/8^
2012-04-06 22:54:31.977 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_s:] []3/*/4/+/4/*/8^
2012-04-06 22:54:31.977 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_p:] [3, 4]3/*/4^+/4/*/8
2012-04-06 22:54:31.978 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_p:] [4, 8]3/*/4/+/4/*/8^
2012-04-06 22:54:31.978 Calculator[1070:207] -[CalculatorViewController parser:didMatchExpr_s:] []3/*/4/+/4/*/8^
2012-04-06 22:54:31.979 Calculator[1070:207] res 0
Why is my res 0?
Source: (StackOverflow)
For an iOS application, I want to parse an HTML file that may contain UNIX style variables for replacement. For example, the HTML may look like:
<html>
<head></head>
<body>
<h1>${title}</h1>
<p>${paragraph1}</p>
<img src="${image}" />
</body>
</html>
I'm trying to create a simple ParseKit grammar that will provide me two callbacks: One for passthrough HTML, and another for the variables it detects. For that, I created the following grammar:
@start = Empty | content*;
content = variable | passThrough;
passThrough = /[^$]+/;
variable = '$' '{' Word closeChar;
openChar = '${';
closeChar = '}';
I'm facing at least two issues with this: for variable I had originally declared it as openChar Word closeChar, but it did not work (I still don't know why). The second issue (and more important) is that the parser stops when it finds <img src"${image}" /> (i.e. a variable inside a quoted string).
My questions are:
- How can I modify the grammar to make it work as expected?
- Is it better to use a tokenizer? If that's the case, how should I configure it?
Source: (StackOverflow)
I'm writing an application where I need to parse wine menu. From what i'ce seen so far, they all follow some structure, the trick wil be defining all thosestructures.
I'm right now exploring using Parsekit and creating grammars, but the learning curve is pretty steep. Rather than spending the next couple weeks figuring it out and then realizing this is not the right approach, I figured I'd ask.
Any insights/resources people would like to share on parsing those kind of things?
Thanks, olivier
Source: (StackOverflow)
I am trying to add the Parsekit framework to my OSX Xcode project. I've never added a 3rd party framework before and I can't get it to work right.
I dragged the included Xcode project into my 'Groups & Files' pane and chose to Add it to my project. I then dragged Parsekit.framework underneath the Link Binary With Libraries heading. Then I double-clicked my target app and added Parsekit as a Direct Dependency. I also added libicucore.dylib as a Linked Library (as it says to do this on their site). Finally, in the Build settings tab of my target info I set the Header Search Paths to /Users/path/to/include/directory and the Other Linker Flags to -ObjC -all_load.
Running this as a debug build work fine with no errors. However, when I build my app to release and then try to run the executable created, the app fails to load with the following error message:
MyApp cannot be opened because of a problem. Check with the developer to make sure myApp works with this version of Mac OS X.
Here is the dump from the crash reporter:
Process: MyApp [11658] Path:
/Users/Garry/Programming/Xcode/Mac/MyApp/build/Release/MyApp.app/Contents/MacOS/MyApp
Identifier: com.yourcompany.MyApp
Version: ??? (???) Code Type:
X86-64 (Native) Parent Process:
launchd [135] Date/Time:
2010-05-24 17:08:08.475 +0100 OS
Version: Mac OS X 10.6.3 (10D573)
Report Version: 6Interval Since Last
Report: 133300 sec Crashes
Since Last Report: 3 Per-App
Crashes Since Last Report: 3
Anonymous UUID:
DF0265E4-B5A0-45E1-8B71-D52A27CFDDCA
Exception Type: EXC_BREAKPOINT
(SIGTRAP) Exception Codes:
0x0000000000000002, 0x0000000000000000
Crashed Thread: 0
Dyld Error Message: Library not
loaded:
@executable_path/../Frameworks/ParseKit.framework/Versions/A/ParseKit
Referenced from:
/Users/Garry/Programming/Xcode/Mac/MyApp/build/Release/MyApp.app/Contents/MacOS/MyApp
Reason: image not found
Model: MacBookPro5,5, BootROM
MBP55.00AC.B03, 2 processors, Intel
Core 2 Duo, 2.53 GHz, 4 GB, SMC 1.47f2
Graphics: NVIDIA GeForce 9400M, NVIDIA
GeForce 9400M, PCI, 256 MB Memory
Module: global_name AirPort:
spairport_wireless_card_type_airport_extreme
(0x14E4, 0x8D), Broadcom BCM43xx 1.0
(5.10.91.27) Bluetooth: Version
2.3.1f4, 2 service, 2 devices, 1 incoming serial ports Network Service:
AirPort, AirPort, en1 Network Service:
Ethernet Adaptor (en6), Ethernet, en6
Serial ATA Device: Hitachi
HTS545025B9SA02, 232.89 GB Serial ATA
Device: HL-DT-ST DVDRW GS23N USB
Device: Built-in iSight, 0x05ac
(Apple Inc.), 0x8507, 0x24400000 USB
Device: Internal Memory Card Reader,
0x05ac (Apple Inc.), 0x8403,
0x26500000 USB Device: IR Receiver,
0x05ac (Apple Inc.), 0x8242,
0x04500000 USB Device: Apple Internal
Keyboard / Trackpad, 0x05ac (Apple
Inc.), 0x0237, 0x04600000 USB Device:
BRCM2046 Hub, 0x0a5c (Broadcom
Corp.), 0x4500, 0x06100000 USB Device:
Bluetooth USB Host Controller, 0x05ac
(Apple Inc.), 0x8213, 0x06110000
After building the app, in addition to the executable file, Xcode is also creating a file called MyApp.app.dSYM. Any idea what that is??
I am developing with Xcode 3.2.2 on an Intel MBP running 10.6.3.
Many thanks for any help offered.
Edit: New problem!
Many thanks to diciu for your suggestions - initially I thought they had solved the problem. However, I copied the compiled app over to my wife's Macbook (Intel, running 10.6.3) but it won't launch. Instead I get the following error message:
You can't open the application MyApp because it is not supported on this type of Mac.
In the Build settings of MyApp, I have Valid architectures set to i386 x86_64 so it should work. What could this be caused by?
Edit: Problem solved
I had inadvertently checked the Build active architecture only flag in the Build settings. Unchecking this fixed my problem. Many thanks!
Source: (StackOverflow)
I am using PK to try and tokenize the .gift file format. I am trying to do the following:
Given a string that is similar to this: @"= 2 + 2"
I am trying to return '2 + 2' without going through the hassle of determining if the token that is going over that string is equal to a symbol and then defining how the output string should be. What I'm trying to do is say that if [PKToken.stringValue isEqualToString: @"="] to pop that value off of the PKTokenizer and then return the rest of the string with formatting still in tact.
Let me know if this was clear enough...
--Skylar.
Source: (StackOverflow)
I was wondering if there were a way to get back how far into an assembly a PKParser has parsed before encountering a syntax error.
reference: http://parsekit.com/
I'm using a grammar that basically describes a prefix notation expression language.
For example:
given your standard prefix notation expression grammar and a string "(+ a - b c))"
I'd like to retrieve that [(,+,a] where matched, so I can give the user some idea of where to look to fix their error, but the completeMatchFor and bestMatchFor don't return anything I can use to find this info.
Ideally I'd like to say that a '(' was expected, but it's not necessary for a grammar as simple as what I'm using.
From the book mentioned as the user manual, it seemed as if I would need to create a custom parser for this, but I was hoping that maybe I'd simply missed something in the framework.
Thoughts?
Source: (StackOverflow)
I am building a parser on top of the TDArithmeticParser.m of ParseKit Tests. I extended the TDArithmeticParserTest.m with the failing test:
- (void)testMath {
s = @"10+(2*3)-15";
result = [p parse:s];
TDEquals((double)1.0, result); // result == 0.0
}
The problem is that I don't understand why the grammar is not working with this test. The corresponding BNF-grammar of the arithmetic parser is:
expr = term (plusTerm | minusTerm)*;
term = factor (timesFactor | divFactor)*;
plusTerm = '+' term;
minusTerm = '-' term;
factor = phrase exponentFactor | phrase;
timesFactor = '*' factor;
divFactor = '/' factor;
exponentFactor = '^' factor;
phrase = '(' expr ')' | Number;
I would be very thankful for any ideas that helps me identifying the problem.
Source: (StackOverflow)
Having got to grips a bit with the ParseKit grammar syntax (playing around in the demo app) I'm now trying to get my own mini demo working, but so far without much success. The assembler callbacks are not getting called.
Below is a condensed version of the relevant code. When testParse runs the parser seems to do it's thing OK and correctly match my string to my anything production (which also works in the demo) but didMatchAnything: is just not getting called.
#import <Foundation/Foundation.h>
@class PKParser;
@interface FileParserThing : NSObject {
PKParser* _parser;
}
- (void)testParse;
@end
#import <ParseKit/ParseKit.h>
#import "FileParserThing.h"
@interface FileParserThing ()
@property (nonatomic, retain)PKParser* parser;
- (void)didMatchAnything:(PKAssembly *)a;
@end
@implementation FileParserThing
@synthesize parser = _parser;
-(id)init
{
if (!(self = [super init])) return nil;
NSString *g = @"@start = anything; anything = Any+;";
self.parser = [[PKParserFactory factory] parserFromGrammar:g assembler:self];
return self;
}
- (void)testParse
{
NSString *s = @"Foo Bar";
NSLog(@"test parse with: %@", s);
[self.parser parse:s];
}
- (void)didMatchAnything:(PKAssembly *)a
{
NSLog(@"Hooray!");
}
@end
Digging around in the ParseKit code I can see that line 129 of PKParser
[assembler performSelector:assemblerSelector withObject:self withObject:a];
Isn't being executed, because assembler is nil. Which, in turn, leads me to the parser factory; where my understanding of what's going on begins to fail.
Disclaimer; I know, I probably need to read The Book, but one thing at a time. I want to get a small proof of concept working, before forking out 30 mice for a book I might never read again if my project is a non-starter :)
Source: (StackOverflow)
I am using Parse Kit and I have question with grammar.
This my grammar string below:
self.grammar = @" \
@start = sentence+; \
sentence = adjectives subjects will verbs subjects '.'; \
subjects = 'i' | 'you' | 'he' | 'she' | 'it' | 'they' | 'we' | 'who else' | 'Apples QA' | 'Hitler' | 'dance';\
verbs = 'eat' | 'sleep' | 'dance' | 'kill' | 'care'; \
will = 'will'; \
adjectives = 'awesome' | 'red' | 'beautiful' | 'odd' | 'useless' | 'temporary';";
So, I suppose that I can create sentence that includes rules of all my terms.
Like in line: "sentence = adjectives subjects will verbs subjects '.';"
So input string from user looks like this: "awesome you will care Apples QA"
I have added delegate (assembler) methods into my assembler like this:
- (void)didMatchSentence:(PKAssembly *)a
- (void)didMatchAdjectives:(PKAssembly *)a
- (void)didMatchWill:(PKAssembly *)a
- (void)didMatchSubject:(PKAssembly *)a
- (void)didMatchVerb:(PKAssembly *)a
but parser invoked just two of them: didMatchAdjectives and didMatchWill
How come?
Source: (StackOverflow)
I'm building a grammar to parse Newick trees using ParseKit for a project I'm working on, and I've gotten this far. It's based on the grammar here: http://en.wikipedia.org/wiki/Newick_format. I'd like to use a grammar for this rather than the existing clunky recursive code I have working now.
However, I'm unsure of how to specify the name and length nodes to account for either empty strings or generalized strings and numbers. I've gotten this far from the examples and on the ParseKit site as well as some skimming of the Bulding Parsers for Java book, but have missed something. Can someone point me in the right direction, please?
Current grammar:
@start = tree+;
tree = subtree ';' | branch ';';
subtree = leaf | internal;
leaf = name;
internal = '(' branchset ')' name;
branchset = branch | branchset ',' branch;
branch = subtree length;
name = *;
length = * | ':' *
Thanks!
--Possible answer:
Maybe these name and length nodes would work. Could anyone confirm?
name = Word | Quoted String;
length = ':' Number;
Source: (StackOverflow)