From d46f997e24844ab481677372a025cceff6646391 Mon Sep 17 00:00:00 2001 From: brettlangdon Date: Fri, 19 Jun 2015 08:50:34 -0400 Subject: [PATCH 1/4] add draft for forge article --- .../about/forge-configuration-parser/index.md | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 contents/writing/about/forge-configuration-parser/index.md diff --git a/contents/writing/about/forge-configuration-parser/index.md b/contents/writing/about/forge-configuration-parser/index.md new file mode 100644 index 0000000..cff4b28 --- /dev/null +++ b/contents/writing/about/forge-configuration-parser/index.md @@ -0,0 +1,76 @@ +--- +title: Forge configuration parser +author: Brett Langdon +date: 2015-06-19 +template: article.jade +--- + + +--- + +## TODO: Fix this starting paragraph +Ok, job is great, lets talk about code! I have always had the aspiration of writing my own +programming language. It is something I never went through as part of my "higher" +education and it has always greatly interested me. I have always made attempts here and +there to learn what I can about how they work and try to deconstruct existing languages or +build what I can of new ones. I have never been very successful, but I am always learning. + +I finally feel that I am making a bit of progress in my learning. Recently I was working +on a project in [go](http://golang.org/) and where I started was trying to determine what +configuration language I wanted to use and whether I tested out +[YAML](https://en.wikipedia.org/wiki/YAML) or [JSON](https://en.wikipedia.org/wiki/JSON) +or [ini](https://en.wikipedia.org/wiki/INI_file), nothing really felt right. What I really +wanted was a format similar to [nginx]() but I couldn't find any existing packages for go +which supported this syntax. A-ha, I smell an opportunity. The project I started is +[forge](https://github.com/brettlangdon/forge). Currently it is still nothing too +impressive, but I feel like it marks a step in my quest for creating a programming +language. + +**Example config file:** +```cfg +# Top comment +global = "value"; +section { + a_float = 50.67; + sub_section { + a_null = null; + a_bool = true; + a_reference = section.a_float; # Gets replaced with `50.67` + } +} +``` + +Effectively what I wrote was a programming language. A very very simple one, but one none +the less a programming language. Basically what this library does is take a configuration +file in the specific format and parses it into an intermediate format. In this case it +ends up being a `map[string]interface{}` but if it were a programming language it could +end up being an [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) +(AST) which is the intermediate representation that some languages use before compiling +the source into machine code or [bytecode](https://en.wikipedia.org/wiki/Bytecode). + +The code itself is comprised of two main parts, the tokenizer (or scanner) and the +parser. The tokenizer turns the raw source code (like above) into a stream of tokens. If +you printed the token representation of the code above, it could look like: + +``` +(COMMENT, "Top comment") +(IDENTIFIER, "global") +(EQUAL, "=") +(STRING, "value") +(SEMICOLON, ";" +(IDENTIFIER, "section") +(LBRACKET, "{") +(IDENTIFIER, "a_float") +(EQUAL, "=") +(FLOAT, "50.67") +(SEMICOLON, ";") +.... +``` + +Then the parser takes in this stream of tokens and tries to parse them based on some known +grammar. For example, a directive is in the form ` +` (where `` can be ``, ``, ``, ``, +``, ``). When the parser sees `` it'll look ahead to the next +token to try and match it to this rule, if it matches then it knows to add this setting to +the internal `map[string]interface{}` for that identifier. If it doesn't match anything +then it has a syntax error and will throw an exception. From 905c4b9edb050f340a1d695575a4ad9b75ffbe93 Mon Sep 17 00:00:00 2001 From: brettlangdon Date: Sat, 27 Jun 2015 13:23:10 -0400 Subject: [PATCH 2/4] finish forge blog post --- .../about/forge-configuration-parser/index.md | 174 ++++++++++++++---- 1 file changed, 141 insertions(+), 33 deletions(-) diff --git a/contents/writing/about/forge-configuration-parser/index.md b/contents/writing/about/forge-configuration-parser/index.md index cff4b28..ae17af7 100644 --- a/contents/writing/about/forge-configuration-parser/index.md +++ b/contents/writing/about/forge-configuration-parser/index.md @@ -1,33 +1,122 @@ --- title: Forge configuration parser author: Brett Langdon -date: 2015-06-19 +date: 2015-06-27 template: article.jade --- +An overview of how I wrote a configuration file format and parser. --- -## TODO: Fix this starting paragraph -Ok, job is great, lets talk about code! I have always had the aspiration of writing my own -programming language. It is something I never went through as part of my "higher" -education and it has always greatly interested me. I have always made attempts here and -there to learn what I can about how they work and try to deconstruct existing languages or -build what I can of new ones. I have never been very successful, but I am always learning. - -I finally feel that I am making a bit of progress in my learning. Recently I was working -on a project in [go](http://golang.org/) and where I started was trying to determine what -configuration language I wanted to use and whether I tested out -[YAML](https://en.wikipedia.org/wiki/YAML) or [JSON](https://en.wikipedia.org/wiki/JSON) -or [ini](https://en.wikipedia.org/wiki/INI_file), nothing really felt right. What I really -wanted was a format similar to [nginx]() but I couldn't find any existing packages for go -which supported this syntax. A-ha, I smell an opportunity. The project I started is -[forge](https://github.com/brettlangdon/forge). Currently it is still nothing too -impressive, but I feel like it marks a step in my quest for creating a programming -language. +Recently I have finished the initial work on a project, +[forge](https://github.com/brettlangdon/forge), which is a +configuration file syntax and parser written in go. Recently I was working +on a project where I was trying to determine what configuration +language I wanted to use and whether I tested out +[YAML](https://en.wikipedia.org/wiki/YAML) or +[JSON](https://en.wikipedia.org/wiki/JSON) or +[ini](https://en.wikipedia.org/wiki/INI_file), nothing really felt +right. What I really wanted was a format similar to +[nginx](http://wiki.nginx.org/FullExample) +but I couldn't find any existing packages for go which supported this +syntax. A-ha, I smell an opportunity. + +I have always been interested by programming languages, by their +design and implementation. I have always wanted to write my own +programming language, but since I have never had any formal education +around the subject I have always gone about it on my own. I bring it +up because this project has some similarities. You have a defined +syntax that gets parsed into some sort of intermediate format. The +part that is missing is where the intermediate format is then +translated into machine or byte code and actually executed. Since this +is just a configuration language, that is not necessary. + + +## Project overview + +You can see the repository for +[forge](https://github.com/brettlangdon/forge) for current usage and +documentation. + +Forge syntax is a file which is made up of _directives_. There are 3 +kinds of _directives_: + +* _settings_: Which are in the form ` = ` +* _sections_: Which are used to group more _directives_ ` { }` +* _includes_: Used to pull in settings from other forge config files `include ` + +Forge also supports various types of _setting_ values: + +* _string_: `key = "some value";` +* _bool_: `key = true;` +* _integer_: `key = 5;` +* _float_: `key = 5.5;` +* _null_: `key = null;` +* _reference_: `key = some_section.key;` + +Most of these setting types are probably fairly self explanatory +except for _reference_. A _reference_ in forge is a way to have the +value of one _setting_ be a pointer to another _setting_. For example: + +```config +global = "value"; +some_section { + key = "some_section.value"; + global_ref = global; + local_ref = .key; + ref_key = ref_section.ref_key; +} +ref_section { + ref_key = "hello"; +} +``` + +In this example we see 3 examples of _references_. A _reference_ value +is one which is an identifier (`global`) possibly multiple identifiers separated +with a period (`ref_section.ref_key`) as well _references_ can begin +with a perod (`.key`). Every _reference_ which is not prefixed with a period +is resolved from the global section (most outer level). So in this +example a _reference_ to `global` will point to the value of +`"value"` and `ref_section.ref_key` will point to the value of +`"hello"`. A _local reference_ is one which is prefixed with a period, +those are resolved starting from the current section that the +_setting_ is defined in. So in this case, `local_ref` will point to +the value of `"some_section.value"`. + +That is a rough idea of how forge files are defined, so lets see a +quick example of how you can use it from go. + +```go +package main + +import ( + "github.com/brettlangdon/forge" +) + +func main() { + settings, _ := forge.ParseFile("example.cfg") + if settings.Exists("global") { + value, _ := settings.GetString("global"); + fmt.Println(value); + } + settings.SetString("new_key", "new_value"); + + settingsMap := settings.ToMap(); + fmt.Println(settingsMaps["new_key"]); + + jsonBytes, _ := settings.ToJSON(); + fmt.Println(string(jsonBytes)); +} +``` + +## How it works + +Lets dive in and take a quick look at the parts that make forge +capable of working. **Example config file:** -```cfg +```config # Top comment global = "value"; section { @@ -40,14 +129,8 @@ section { } ``` -Effectively what I wrote was a programming language. A very very simple one, but one none -the less a programming language. Basically what this library does is take a configuration -file in the specific format and parses it into an intermediate format. In this case it -ends up being a `map[string]interface{}` but if it were a programming language it could -end up being an [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) -(AST) which is the intermediate representation that some languages use before compiling -the source into machine code or [bytecode](https://en.wikipedia.org/wiki/Bytecode). - +Basically what forge does is take a configuration file in defined +format and parses it into what is essentially a `map[string]interface{}`. The code itself is comprised of two main parts, the tokenizer (or scanner) and the parser. The tokenizer turns the raw source code (like above) into a stream of tokens. If you printed the token representation of the code above, it could look like: @@ -68,9 +151,34 @@ you printed the token representation of the code above, it could look like: ``` Then the parser takes in this stream of tokens and tries to parse them based on some known -grammar. For example, a directive is in the form ` -` (where `` can be ``, ``, ``, ``, -``, ``). When the parser sees `` it'll look ahead to the next -token to try and match it to this rule, if it matches then it knows to add this setting to -the internal `map[string]interface{}` for that identifier. If it doesn't match anything -then it has a syntax error and will throw an exception. +grammar. For example, a directive is in the form +` ` (where `` can be +``, ``, ``, ``, ``, +``). When the parser sees `` it'll look ahead +to the next token to try and match it to this rule, if it matches then +it knows to add this setting to the internal `map[string]interface{}` +for that identifier. If it doesn't match anything then it has a syntax +error and will throw an exception. + +The part that I think is interesting is that I opted to just write the +tokenizer and parser by hand rather than using a library that converts +a language grammar into a tokenizer (like flex/bison). I have done +this before and was inspired to do so after learning that that is how +the go programming language is written, you can see here +[parser.go](https://github.com/golang/go/blob/258bf65d8b157bfe311ce70c93dd854022a25c9d/src/go/parser/parser.go) +(not a light read at 2500 lines). The +[scanner.go](https://github.com/brettlangdon/forge/blob/1c8c6f315b078622b7264b702b76c6407ec0f264/scanner.go) +and +[parser.go](https://github.com/brettlangdon/forge/blob/1c8c6f315b078622b7264b702b76c6407ec0f264/parser.go) +might proof to be slightly easier reads for those who are interested. + +## Conclusion + +There is just a brief overview of the project and just a slight dip +into the inner workings of it. I am extremely interested in continuing +to learn as much as I can about programming languages and +parsers/compilers. I am going to put together a series of blog posts +that walk through what I have learned so far and which might help +guide the reader through creating something similar to forge. + +Enjoy. From e74f6b0e7cec25b1a487b4d0f1f134c5e5bc1cf4 Mon Sep 17 00:00:00 2001 From: brettlangdon Date: Sat, 27 Jun 2015 13:23:28 -0400 Subject: [PATCH 3/4] give 'li code' the same styles as 'p code' --- contents/css/main.css | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contents/css/main.css b/contents/css/main.css index 225f8d9..ee15e85 100644 --- a/contents/css/main.css +++ b/contents/css/main.css @@ -317,7 +317,7 @@ pre code { line-height: 1.1; } -p code { +p code, li code { padding: 0.1em 0.3em 0.2em; border-radius: 0.3em; position: relative; From a2858fb1679ab187b57139112a7eebfa47438037 Mon Sep 17 00:00:00 2001 From: brettlangdon Date: Sat, 27 Jun 2015 13:25:41 -0400 Subject: [PATCH 4/4] remove some social icons --- contents/css/main.css | 2 +- contents/img/geeklist.png | Bin 1767 -> 0 bytes contents/img/googleplus.png | Bin 1681 -> 0 bytes contents/img/pandora.png | Bin 1586 -> 0 bytes templates/mixins/social.jade | 6 ------ 5 files changed, 1 insertion(+), 7 deletions(-) delete mode 100644 contents/img/geeklist.png delete mode 100644 contents/img/googleplus.png delete mode 100644 contents/img/pandora.png diff --git a/contents/css/main.css b/contents/css/main.css index ee15e85..c940d83 100644 --- a/contents/css/main.css +++ b/contents/css/main.css @@ -450,7 +450,7 @@ code.lang-markdown .bullet { } section.social-links { - width: 304px; + width: 191px; margin-left: auto; margin-right: auto; } diff --git a/contents/img/geeklist.png b/contents/img/geeklist.png deleted file mode 100644 index 71c07b8176004292c1b6ae2be821ee4251bc03f4..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1767 zcmZ`(X*}DB7X6DgN^40Il_Hi>R4bNgq$Y?&l3J!!wKF32HEJuG+Mh)2duSikXrnac zS!$SK7`2m{z8a;JqNA3Idejmoec#@P_q+GpbI-l^e7%fob_8ijMM(evq)9}ay?_zl zAt5HHk@4}{0uu4Ju(bey#@rK!WKn?@inJ$S0nXUjmG74nqP;Bu#A^aTQW^m43r0zQ z0YJ1q0DSfW0CX7uz^M1zubT=MU@FliQozLj!fG%WBWT3DtqC~b_hswM%p*2i(RPYc0@x|S~;PUVY?eY{?jmaIc8@k%=*D-X~j!{sEF*0 zT+ZtAp4dTSv&*u})VnIY4=-STIM4m2T`}m>RCBzmo+4Wzd!0_i87=7IPz$PoA$jxD zD~z%=*EMf#HR*X=H)IoWmL6;3^3gPzqyU92-&Kfs+d$9UEH}^JUztj$=Up(-MKFMS zU71wgAf%Iy&K7mRIOf0dBfc%OQR`lC`_|v7K`4tY#|tAyG88@?ClXA(G{_nVktRms zVqYH(j339^^K^(8J=ewK`(fp82*ju~M7@z+9wKx`3+$jy_G&q6p~7b19iww;qKD4V zlO3VMh*m0iO~*aWS?UV!m+4M-H}=5LECU0|e($t(mQ_^mJEv*uywE^yf-`KMPZC@I0c!p%*7O{Qk-r ztjQ)6=Y${@J=m5<0jWFI5;$3AL_zyjy^AiP(MKe$wvE9SP4$aYkH? z2gi$+5oU#8sP4?J!c$Y1R#j(}Z7$}4S#WTO{1A+cTc}o|GX_d|?0XOh4E&77cF-ck zBN}T;Oiv+2f3hh4T0I-fAzx|oGibuGsg^J35*8nti>L}haygN0jE?RT;zze_oiHvb zGpg?f%X1a9)nGR^HQxNkADAG zrVZ=ysH_D&`pC~oVeTQz35|os2FVnI4vl`D?Pz_SW8V@!k@}uiNt%Kmqcp$BSTp$)0@^5Bs{ImimBPF5lLnu&boBc!_wqA>q29Gn_k z)wa_O&WgB-o}At#qkdS$vMjZmj)GB+eB}auN{ydvrw9AN?E}NDz%uno5vXNy-Z>jQ zx+;}mhcdjLWHr_+K|$3W8g28Qx<0qa5*;+fnVqsoAF=AxLMoD^62!QZrAW|Ie!*#D zv>xM*YoC}6Zg}zO`Js9QvcxAi&Zq$t3Cs1>F&KNBSj%h=GGvu-DBd+aOCLrj=cC^H zZ^D=P+vT|~-wNPJk(+_M8;{cild8AzX%)esVtBQEP);u?(0eG)M@g@_$9b%($eOmj z7|p`=Wjma?H*L>lO_%}i@j;BnoS!o8&X{%ZE~K=;&U23=MeVEL@wsa$CNMI-p73kx zpWViWDiL2GFt_2Xa&W#$@02#FR*ePgt~K5t#3<$nyJ;XZ+WasS_t)X)GH7yuI94#&ZIrTh)M CX&P|= diff --git a/contents/img/googleplus.png b/contents/img/googleplus.png deleted file mode 100644 index a36053f2ba6b61fe9ea742d54d657d005f8b0c34..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1681 zcmV;C25$L@P)9hxB&loba6005H}2cak$tT!p1Djk_0 z600vEt1~5@Eg+N~5R)Acog)>eDjSv_5u_*?sw*9DIBXS9nh4HUU=;_&)kL&I3$bEF)pP0f(JL2c*=k@ozQAWvLQPY8VyD=is zpPIBJ6WZC^(Q|3xot4gHSG`wE#)Ev=sifuk{l1EYy<%D3+}^}sRqe;Z+Pb&h#lz^y z#jiCe+~MNOLNeOFyUC-V(s*s>s-n%cwaBfk+l_?TSxVbfL&%qt&}dxMWmmQ~DBMju z;8{rT`TEX)cjm~%xIi<{UQzAY*X7*X%64q%!MxRRW%~dB!9Fk4#l^vMYv7fP&a|@O ziGI;qOubG)-ng^kfOE-{kJ_=V(#*}UGbYnjMz16l*?e-*Tu-_@F1RWiwNgjfTTJ6; zSKee)+h9=ORz=-jPT^oq;af}JQb5l{Gxq=gz(h9i>+9O2p4FU|$52A}`~2pxs^0GL z#!^MVG$Y@6Y}$8g+P1U9Hz&naN#Sf=)nZfHPd&M1UDI4n(_m5JWK+UULE1|>)=xgn zKQ6d2A>+Nd%u__iS4+^Urne^-<)xj?Og+_DN!43S#z{NPN;NW)P?*fnyIKDV3%St=6Fd@uA zFwsXg#Y;WWLo(Y}M!h#E#yu>{KrhTiG|oXVz&I(pH73G3E6G1ExilrQFCpDqO2a!V z$UZK`JS^K&LfTM2+f_u`QbF2IJl;+`*-bmgJuTlpS;7DS03CEvPE-8*+2Q)#{@LOD z`T1o4X8QU0`T6xO#a%-*Z$EZ!)TGQr|8&d~! zqZH53I#f6^Jcb<LMSrR&z6cJ7X$I`-<&79q&_yz}ZFyrwy%d-_SCKT5l=S%BdGh6E&mb9b1U&f&qzb zlbR*;VH@l}<1SlihmrJ*8)rOK*}+SHW~r>Byn5HYMWP z(0t%=X8EqQ|G5D*TDN)C1kjGImpF{lGd4l-ALi~rq&#xy;DHa~Kb`)((@Ewy>_FpU z7y;%?B2|0W6h0lbH2nUN-AMUq(XyFDUjz^W1SZ^>la|cZuAaeLPks5x7o90TkgcTe z)dT`+EHL)Qiwy%uw&!a(2T?x;fG}*IG9cS`aadD56mp8`~eR-)Af9lC7cIbmgJ|!4QkSdg#Wkt7;H;=V!Y| zf)9yb5fEK;mG>Bf2HqibPvA-UVdJMV>qc|~s~W4Az$E&t^}aAXDL<8N(E!0>qQ(Lf zidMHMY@T@+}2P)vVfRCSeo4f!307h_| zFkp`_U5(Aw-!EQ{xW?4!>+HoDPNSl$=AWm+3M@;Ws9(?yw3mp_-c-^O?jj+ zWtaB!^2^rX_44r5+~O-&gTl_)pSI1ZyUwk@(4e-<)7s*;#MJrq@{Om#zRcL%;pQw~ zlhE1W(%RuHT8XQ@&v}@+`S$ei@a~PM!Z&G}D_)Yd!p`C3=&riQF<*~!i=xKS+%sT~ z$I{!9o1`*ekH5^=h@`)cn4*cLz{%6yrMS%f`t`8E(zwUf`S*wz8?<-)HBU+L; zYMbor?I~W9DPWZC?(U1G!E=?iLUEuXT9AgKzL2THEMb-*S&>?Tsw-iYWQwmNT9HzG zr(A@rR)DB4W0&~$@+V)DS%Ind^zkrXj?CBL^6~Gv$ky}n@AmZZ`S$b6)!y#z?c(L= zJZ+q|#ni#g+34x(grU8cuf~3yyY%z$owLdA?d~U3g8BCI`1SJf@9**Q@%#AoBwUhY zimvtZ@Frc8;^yfkU6O8)vo2zmWs9$mslxg7^UT)Z^YQTd_Ve)X?)&%i_w@4g^YQZW z@a^sG{P^@JU6c3q@$m5P^6>A>*Wk<6;P3A4_x1Ae@bB;M?kHZ9FJqSP@9!;OlETi~ z^YZZY^6>TZ@%s1lFkp@?V3RChk~CqClA5IV_4BmB&F=2*wB`tt0000abW%=JhJ_(L zAqdkw2-Ap$H@1^fBIyyTgiQ;Sn1FHh7LF8>hkgK(0TyaP0008LNklAvpY7@)gPT2>CEK!)>_-NfEM7@&8e>nS!FB?d7GN6(qZ zzaIYv0}sD_g@8*logF2_7(};qI-PU>`2XQ|IN0;(KM;JJbOd+A{P zu?dqVO*nGoh{M7Cd!B=o{BLM)5n&Mj)%q96el+>SwF?(6&z@`VvUcT?qm5ugHnslZ zV^IF}52Rq>m-|l^zhB^W*yD+it|z6V@hu+<5coMeO3IOV=mA#j0Rq)P@b~;$p5x zT)FfZtAY;)R&CjmezWW07qAGLf~_DcHhulF^?TOG|JW2f`26AhnupInzWWaYXaV&g z>H7QlT?lE6a5$5=I${0WkGK`+rfy!5G4Cye^B7&h<1bIXUwfaK*`Idn9-0XdH{9tt znzHu7)s=IXO^JSvso?eh9w;5CK z-<$XL@x2$1AH9AIHbMN?zw_pnRzz1R+EZM*Q>U<`zW+sek{)b zMD1w)-mc=q#^tsA3_`-q)%U;Vym+wV#n-zzckY$l+O==fj&HjfzkSiIZWb0|P+}51 zP>}udW%litx3jZv1M&9l*)MOue7QZl;D8_tDDw*N=BfSwfo_y-KAsORwtfOEH! k5)U_*0tVpX<^jn80LtgtZxox;iU0rr07*qoM6N<$g7^}$Q2+n{ diff --git a/templates/mixins/social.jade b/templates/mixins/social.jade index 7ac534f..53af958 100644 --- a/templates/mixins/social.jade +++ b/templates/mixins/social.jade @@ -4,16 +4,10 @@ mixin link_img(type, link, image) mixin social_links ul - li - +link_img("Google+", "//plus.google.com/100527884089547887285/", "/img/googleplus.png") li +link_img("Twitter", "//www.twitter.com/#!/brett_langdon", "/img/twitter.png") li +link_img("Linkedin", "//www.linkedin.com/in/brettlangdon", "/img/linkedin.png") - li - +link_img("Pandora", "//www.pandora.com/#!/profile/activity/brett1129", "/img/pandora.png") - li - +link_img("Geeklist", "//geekli.st/brett_langdon", "/img/geeklist.png") li +link_img("GitHub", "//github.com/brettlangdon", "/img/github.svg") li