Nightmare scenario
2020-04-03 Filed in: programming
My dream of the perfect software development approach
has one, eh, tiny drawback: it's a
nightmare. Let me explain.
The Vlerq project is about very high level concepts as well as the close-to-raw-silicon implementation. After nearly four decades of exposure to computing technology, I consider Lisp the most powerful production-grade environment available today for taking very high level concepts to a runnable form. A system such as SBCL combines everything into one system which bridges an extra-ordinary range of interactivity (through Slime and Emacs), excellent introspection and debugging, and machine-code generation, all in one. I envy the masters who know how to live inside SBCL and are able to perform magic in a world which brings together extreme abstraction and raw performance.
Problem #1 is that SBCL is not deployable, let alone embeddable in other languages (a separate process is tricky for robust deployment, and SBCL is a pretty large environment to drag along). For something as general-purpose as Vlerq, being tied to a little-used system is a big issue. Problem #2 is that I have far too little experience with both SBCL and Emacs to be truly productive with them in the next years (yes, it takes years to make app, code, and bits sing, in my experience).
Problem #1 could be addressed by generating code for another system, such as the (delightfully Scheme-like) Lua language, along with C-coded primitives for all performance-specific loops and bit-twiddling. But that means problem #2 will sting even more: now I not only have to become proficient with SBCL, I also need to implement complex code on multiple levels, so that the generated Lua + C source code flies well.
It gets worse: problem #3 is that Lua is not really rich enough yet in terms of application-level libraries. In fact, I consider Tcl to be one of the best application-level languages around (yes, above even Python and Ruby) because of its excellent malleability and the way it supports domain-specific languages. Like Lisp, Tcl melds code and data together in a very fluid way - meaning it allows you to bring the design towards the app, instead of the other way around. It's ironic that Lisp and Tcl share this property, but at totally opposite ends: Lisp on the algorithmic side and Tcl on the application glue side.
So what's the nightmare? Well, to be truly effective in such a context, I'd have to be a master in SBCL, Emacs, Lua, C, and Tcl. And I'm not. I've been feeling the pain for a decade now. And I just don't know whether I should aim for such a context, or just muddle along in one or two technologies - with all the limitations associated with them.
I'd love to be proven wrong, but those who say "use language X" probably don't understand what breadth of conceptual / performance gap I'm trying to bridge. In a way that works outside a laboratory setting, that is.
The Vlerq project is about very high level concepts as well as the close-to-raw-silicon implementation. After nearly four decades of exposure to computing technology, I consider Lisp the most powerful production-grade environment available today for taking very high level concepts to a runnable form. A system such as SBCL combines everything into one system which bridges an extra-ordinary range of interactivity (through Slime and Emacs), excellent introspection and debugging, and machine-code generation, all in one. I envy the masters who know how to live inside SBCL and are able to perform magic in a world which brings together extreme abstraction and raw performance.
Problem #1 is that SBCL is not deployable, let alone embeddable in other languages (a separate process is tricky for robust deployment, and SBCL is a pretty large environment to drag along). For something as general-purpose as Vlerq, being tied to a little-used system is a big issue. Problem #2 is that I have far too little experience with both SBCL and Emacs to be truly productive with them in the next years (yes, it takes years to make app, code, and bits sing, in my experience).
Problem #1 could be addressed by generating code for another system, such as the (delightfully Scheme-like) Lua language, along with C-coded primitives for all performance-specific loops and bit-twiddling. But that means problem #2 will sting even more: now I not only have to become proficient with SBCL, I also need to implement complex code on multiple levels, so that the generated Lua + C source code flies well.
It gets worse: problem #3 is that Lua is not really rich enough yet in terms of application-level libraries. In fact, I consider Tcl to be one of the best application-level languages around (yes, above even Python and Ruby) because of its excellent malleability and the way it supports domain-specific languages. Like Lisp, Tcl melds code and data together in a very fluid way - meaning it allows you to bring the design towards the app, instead of the other way around. It's ironic that Lisp and Tcl share this property, but at totally opposite ends: Lisp on the algorithmic side and Tcl on the application glue side.
So what's the nightmare? Well, to be truly effective in such a context, I'd have to be a master in SBCL, Emacs, Lua, C, and Tcl. And I'm not. I've been feeling the pain for a decade now. And I just don't know whether I should aim for such a context, or just muddle along in one or two technologies - with all the limitations associated with them.
I'd love to be proven wrong, but those who say "use language X" probably don't understand what breadth of conceptual / performance gap I'm trying to bridge. In a way that works outside a laboratory setting, that is.