Text
Page: 1
mruby de Hello World!
HASUMI Hitoshi @hasumikin
Monstar Lab, Shimane office
富山Ruby会議01
2019-11-03
Page: 4
Today is
My birthday 🎂
Page: 5
me
HASUMI Hitoshi
@hasumikin
Microcontroller
detective
RubyKaigi 2018, 2019
RubyWorld 2018
KRKRB 2019 (Poland)
RubyConf 2019 (the US)
Page: 6
RubyWorld Conference 2019
Nov. 7-8, 2019 / Matz江, the Holy City of Ruby + 🍶
Page: 7
RubyWorld Conference 2019
🍶熱燗 sponsored
by Monstar Lab
Page: 9
Monstar Lab
We don't have Toyama office, though
Page: 10
Monstar Lab
@noboru_i
Page: 12
Monstar Lab
いわゆるゴッド
Page: 13
Monstar Lab
この人です
Page: 14
Monstar Lab
WE ARE HIRING!
Page: 15
Monstar Lab
採用してます
Page: 16
mruby de Hello World!
How to code
Page: 17
mruby de Hello World!
5249
545a
3032
1001
0000
0000
0008
5445
3030
0000
4f02
0c48
0100
3030
3030
0060
002e
656c
0470
3036
4952
0001
0100
6c6f
7574
9a78
4550
0004
0137
2057
7300
0000
0000
0000
0167
6f72
454e
0062
0044
0000
0000
6c64
4400
4d41
3030
000c
0001
2100
0000
Page: 18
puts "Hello World!"
5249
545a
3032
1001
0000
0000
0008
5445
3030
0000
4f02
0c48
0100
3030
3030
0060
002e
656c
0470
3036
4952
0001
0100
6c6f
7574
9a78
4550
0004
0137
2057
7300
0000
0000
0000
0167
6f72
454e
0062
0044
0000
0000
6c64
4400
VM code
4d41
3030
000c
0001
2100
0000
RITE0006.x...bMA
TZ0000IREP...D00
02...`..........
..O......7.g....
...Hello World!.
.....puts.END...
..
Page: 19
CRuby, mruby and mruby/c
Page: 20
CRuby(2.6.4) + Ruby code
puts "Hello World!"
rss = `ps -o rss= -p #{Process.pid}`.to_f / 1024
vsz = `ps -o vsz= -p #{Process.pid}`.to_f / 1024
puts "RSS: #{rss} MB"
puts "VSZ: #{vsz} MB"
#
#
#
#
$ ruby hello.rb
Hello World!
RSS: 13.63671875 MB
VSZ: 78.6328125 MB
Page: 21
mruby(2.0.1) + Ruby code
#include <mruby.h>
#include <mruby/compile.h> // compile at runtime
int main(void) {
mrb_state *mrb = mrb_open();
char code[] = "puts 'Hello World!'";
mrb_load_string(mrb, code);
mrb_close(mrb);
return 0;
}
// $ valgrind ./hello_ruby
// (...)
// Hello World!
// ==18802==
// ==18802== HEAP SUMMARY:
// ==18802==
in use at exit: 0 bytes in 0 blocks
// ==18802== total heap usage: 3,067 allocs, 3,067 frees,
//
379,851 bytes allocated
Page: 22
mruby(2.0.1) + VM code
#include <mruby.h>
#include <mruby/irep.h>
#include "hello.c"
// compiled by mrbc
int main(void) {
mrb_state *mrb = mrb_open();
mrb_load_irep(mrb, hello);
mrb_close(mrb);
return 0;
}
// $ valgrind ./hello_vm
// (...)
// Hello World!
// ==18858==
// ==18858== HEAP SUMMARY:
// ==18858==
in use at exit: 0 bytes in 0 blocks
// ==18858== total heap usage: 3,057 allocs, 3,057 frees,
//
329,083 bytes allocated
Page: 23
mruby/c(2.0) + VM code
#include "mrubyc/src/mrubyc.h"
#include "hello.c"
#define MEMORY_SIZE (1024 * 12) // RAM:12KB
static uint8_t my_memory_pool[MEMORY_SIZE];
int main(void) {
mrbc_init(my_memory_pool, MEMORY_SIZE);
mrbc_create_task(hello, 0);
mrbc_run();
return 0;
}
Page: 24
Hello memory usage
ROM
RAM
ROM
/ RAM
CRuby + mruby + mruby + mruby/c
Ruby
Ruby VM code + VM
code
code
code
18MB(*) 3561KB 2736KB
148KB
13MB
370KB
321KB
12KB
1.3
9.6
8.5
12.3
(*)...binary size of `bin/ruby` itself
Page: 25
CRuby, mruby and mruby/c
Page: 26
mruby compiler
How to code?
Page: 27
Steps of coding a compiler
Tokenize (Scan, Lexical analyze)
Parse
Generate Code
Not detailed enough 😵
Page: 28
Steps of coding a compiler
Tokenize (Scan, Lexical analyze)
Find keywords
Classify tokens
Parse
Make syntax tree
Make symbol table
Make literal pool
Count local variables and registers
Make each scopes （文字数
Page: 29
Steps of coding a compiler
Tokenize (Scan, Lexical analyze)
Parse
Generate Code
Just outlines for today
Page: 30
mruby compiler written in CRuby
github.com/hasumikin/mmrbc.gem
Page: 31
mruby compiler written in CRuby
github.com/hasumikin/mmrbc.gem
for only `puts "Hello World!"` 😂
`[identifier] "[string literal]"`
Page: 32
mruby compiler written in CRuby
github.com/hasumikin/mmrbc.gem
for only `puts "Hello World!"` 😂
`[identifier] "[string literal]"`
but no cheat 😤
Page: 33
Tokenizer
FLEX is a tokenizer generator
You can write tokenizer from scratch w/o FLEX
CRuby, mruby and mmrbc.gem, too
puts("Hello World!")
....................
p
pu
put
puts
puts( # look ahead
puts
# detemine a token
Page: 34
Tokenizer of Ruby
It has state
$ irb
irb(main):001:0> [1, 2, 3].each do |n|
irb(main):002:1*
`do` keyword sets tokenizer_state as EXPR_BEG
irb can delay parsing until it becomes
EXPR_END with `end` keyword
Page: 35
Parser and Parser generator
Parser
Syntactic analysis of token list
Parser generator
Generates C code of parser by syntactic definition
(and "reduction" code)
Parse algorithms
LL(n), LR(n), etc.
Page: 36
Parse algorithm - LL(1)/LR(1)
LL(k) = Left to right, Leftmost derivation
You can write LL parser from scratch
LR(k) = Left to right, Rightmost derivation
You can hardly write LR parser from scratch
You should use parser generator
LALR(k) is a variation of LR
(k) = length of lookahead symbols
Page: 37
Parser generator
YACC/BISON
Page: 38
YACC/BISON
Most popular parser genarator
Used in CRuby, mruby, bash, Blawn, etc.
BISON is a GNU version of YACC
Thread safe (Reentrant)
Genarates LALR(1) parser
Page: 40
However
I don't use YACC/BISON
Page: 42
So?
LEMON, instead
Page: 43
LEMON?
Parser generator of SQLite
A part of SQLite project
Generates LALR(1) parser code
as well as YACC/BISON does
Doesn't use global variable to pass information
between parser and tokenizer
YACC/BISON does
Tokenizer calls parser in LEMON
Parser calls tokenizer in YACC/BISON
Page: 44
Parser calls tokenizer in YACC/
BISON
int yyparse(parser_state *p) {
...
yynewstate:
...
yychar = yylex (&yylval, p); // calls tokenizer
...
goto yynewstate;
...
}
Page: 45
Tokenizer calls parser in LEMON
void Tokenize(char *code) {
...
while (token == get_token(code)) {
...
Parse(parser, token, value); // calls parser
...
}
Parse(parser, 0, NULL);
}
Page: 46
Parsing "Hello World!" in YACC
# an excerpt from mruby/mrbgems/mruby-compiler/core/parse.y
primary
: literal
| string
(...)
;
literal
: numeric
(...)
;
string
: string_fragment;
| string string_fragment
{ $$ = concat_string(p, $1, $2); }
;
string_fragment : tSTRING_BEG string_rep tSTRING
{ $$ = new_dstr(p, push($2, $3)); };
string_rep
: string_interp
| string_rep string_interp
{ $$ = append($1, $2); };
string_interp : tSTRING_MID
{ $$ = list1($1); };
Page: 47
Parsing "Hello World!" in LEMON
# an excerpt from mmrbc.gem/ext/mmrbc/parse.y
primary
::= literal.
primary
::= string.
literal
::= numeric.
string
::= string_fragment.
string_fragment(A) ::= STRING_BEG string_rep(C) STRING.
{ A = new_dstr(p, list3(atom(ATOM_string_add),
list1(atom(ATOM_string_content)), C)); }
string_rep
::= string_interp.
string_rep(A)
::= string_rep(B) string_interp(C).
{ A = append(B, C); }
string_interp(A)
::= STRING_MID(B).
{ A = list2(atom(ATOM_at_tstring_content),
literal(B)); }
Page: 50
Future work ->{mmrbc + mruby/c}
More syntax than `puts "Hello World!"`
Rewrite mmrbc.gem into C
And embed it with mruby/c in one-chip
microcontroller which has less than 512KB ROM
and 128KB RAM
Page: 51
Future work ->{mmrbc + mruby/c}
Page: 52
Future work ->{mmrbc + mruby/c}
Page: 53
Future work ->{mmrbc + mruby/c}
"LEMON would generate smaller binary than YACC."
Page: 54
Future work ->{mmrbc + mruby/c}
Possibly and hopefully,
I will see you on
RubyKaigi 2020 at Matsumoto
