Javascript (and SemWeb) Dev on Linux

tl;dr - KDevelop + Terminator + UglifyJS + Rapper + Fuseki + cURL + Chrome

Usually the majority of my coding has been in Java, and I've generally used the Eclipse IDE. For the Stanbol stuff I've got NetBeans. Both of these are great for Java. But my Seki project uses node.js server-side, and quite a lot of Javascript client-side, together with HTML, CSS and RDF formats. I wound up having Eclipse configured with a project/file nav pane, one for source and 3 terminal consoles. It was ok, but seemed overkill, not very responsive, and recently became somewhat broken. I made a start on a dedicated custom editor (JEdwards) which I must get back to one day. Meantime I've played with a few alternatives. The best setup I've found so far is to use KDevelop for nav & editing, with terminator for the shell consoles, both available in the regular Ubuntu repositories.

Code Editing

KDevelop is a native code IDE, primarily focussed on C/C++ but supports lots of other languages. It's what you'd expect, and nice and responsive. I'm running standard Ubuntu (with Gnome Classic) so KDevelop needed a pile of KDE dependencies, but after it's installed them, works a treat. One minor irritation is that KDevelop doesn't have a formatter/indenter for Javascript. A solution here is UglifyJS. It's available though npm install uglify-js@2 but an earlier, perfectly functional version is available from Ubuntu repositories, as node-uglify. I'm currently just running the beautify command uglifyjs -b --overwrite {filename}, though I must write a little script to act on directories and also create backup files just in case. Oops,changed my mind - I hadn't noticed it does quite a few idiomatic shortenings, as far as I'm concerned obfuscating the code. js-beautify seems a much more practical bet. npm -g install js-beautify

Terminals

Solution here is Terminator. I reckon is a must-have if you use consoles a lot. It simply allows you to run several of them in the one window (I have 3, split vertically), much less hassle than having a load of seperate windows floating around. Tip : <Ctrl>+<Shift>+g clears the current console.

Client-Side

cURL, the well-know command line HTTP tool is really handy (is in Ubuntu repos), but for all the user-friendly bits, the Google Chrome browser's dev bits seem the best, is really quite sophisticated. (There is also the open source version Chromium, but I've yet to it - suspect it's fairly indistinguishable).

Working with RDF

Seki makes major use of RDF, so I need tools for that as well. I've been working on a dedicated RDF IDE/hacking toolkit (Scute) on and off for a couple of years now, but it's still far from usable. There are a few integrated tools around, I guess the nearest open source thing out there is Protege, but that's more oriented toward ontology dev. So for now I've just using Turtle syntax, text-editing in KDevelop, with Rapper for conversion. Seki is designed to have a SPARQL 1.1 server as back end, and during dev I'm using Fuseki (Jena) which comes bundled with the usual kind of online query form which is adequate there [just remembered I did get the SPARQL client bits of Scute working to a useful level, must see where I was up to...]. It's very likely I'll make some more renderings/forms to make life easier there.

Other suggestions very welcome.

Comments to G+


danja
2013-06-07T01:09:19+01:00
kdevelop js ide dev tools development turtle semweb rdf rapper javascript terminator utilities
Related
Comments
Edit

Javascript (and SemWeb) Dev on Linux

tl;dr - KDevelop + Terminator + Rapper + Fuseki + cURL + Chrome

Usually the majority of my coding has been in Java, and I've generally used the Eclipse IDE. For the Stanbol stuff I've got NetBeans. Both of these are great for Java. But my Seki project uses node.js server-side, and quite a lot of Javascript client-side, together with HTML, CSS and RDF formats. I wound up having Eclipse configured with a project/file nav pane, one for source and 3 terminal consoles. It was ok, but seemed overkill, not very responsive, and recently became somewhat broken. I made a start on a dedicated custom editor (JEdwards) which I must get back to one day. Meantime I've played with a few alternatives. The best setup I've found so far is to use KDevelop for nav & editing, with terminator for the shell consoles, both available in the regular Ubuntu repositories.

Code Editing

KDevelop is a native code IDE, primarily focussed on C/C++ but supports lots of other languages. It's what you'd expect, and nice and responsive. I'm running standard Ubuntu (with Gnome Classic) so KDevelop needed a pile of KDE dependencies, but after it's installed them, works a treat. One minor irritation is that KDevelop doesn't have a formatter/indenter for Javascript. UPDATE: decided I don't like the code changes Uglify makes (it's less readable), so scratch that! A solution here is UglifyJS. It's available though npm install uglify-js@2 but an earlier, perfectly functional version is available from Ubuntu repositories, as node-uglify. I'm currently just running the beautify command uglifyjs -b --overwrite {filename}, though I must write a little script to act on directories and also create backup files just in case.

Terminals

Solution here is Terminator. I reckon is a must-have if you use consoles a lot. It simply allows you to run several of them in the one window (I have 3, split vertically), much less hassle than having a load of seperate windows floating around. Tip : ++g clears the current console.

Client-Side

cURL, the well-know command line HTTP tool is really handy (is in Ubuntu repos), but for all the user-friendly bits, the Google Chrome browser's dev bits seem the best, is really quite sophisticated. (There is also the open source version Chromium, but I've yet to it - suspect it's fairly indistinguishable).

Working with RDF

Seki makes major use of RDF, so I need tools for that as well. I've been working on a dedicated RDF IDE/hacking toolkit (Scute) on and off for a couple of years now, but it's still far from usable. There are a few integrated tools around, I guess the nearest open source thing out there is Protege, but that's more oriented toward ontology dev. So for now I've just using Turtle syntax, text-editing in KDevelop, with Rapper for conversion. Seki is designed to have a SPARQL 1.1 server as back end, and during dev I'm using Fuseki (Jena) which comes bundled with the usual kind of online query form which is adequate there [just remembered I did get the SPARQL client bits of Scute working to a useful level, must see where I was up to...]. It's very likely I'll make some more renderings/forms to make life easier there.

Other suggestions very welcome.

Comments to G+


danja
2013-06-06T10:24:13+01:00
kdevelop js ide dev tools development turtle semweb rdf rapper javascript terminator utilities
Related
Comments
Edit

Introducing JEdwards

JEdwards is a little sub-project I've just been putting together in Java. Screenshot.

It's so named for two reasons:

  1. it's roughly a contraction of "towards a Javascript editor"
  2. it's something you probably want to ignore (like twincest :)

Having said that, it does have a couple of features that may be of interest to sane developers:

  1. a Java terminal emulator (bash shell)
  2. syntax highlighting for SPARQL/Turtle

Neither are entirely finished, but both are useable/reusable (Apache 2 license, or somesuch).

evil jedward

I've been using Eclipse for most of my dev stuff for years now. When I was doing things in Node.js I wound up configuring it to have a file explorer pane, a text editor pane (for Javascript, HTML, Turtle or SPARQL) and three terminal panes all connected to the local shell. Eclipse was basically a (slow) sledgehammer to crack a nut. I did spend a while looking for a way of setting these things up using separate apps, but was beaten by the problem of pinning the windows to the workspace. I believe it should be possible using Devil's Pie or similar, but I had no joy. But as it happened I wanted a terminal emulator in Java anyhow and had played with syntax highlighting before.

In Scute I'd put together some basic highlighting for Turtle, except when I came to look at it again it was a bit too hardcoded to reuse, and Javascript is quite complicated... Looking around I came across jsyntaxpane, which is a pluggable highlighter which takes its config from a JFlex lexer. It'd got the necessary for Javascript, so I decided to use that instead of my hacky code. I found a SPARQL/Flex file on the Web that someone had prepared for IntelliJ IDEA which although was geared to do other things saved me a bit of time writing out the SPARQL patterns. Here's sparql.flex.

For the terminal emulator I started with the JConsole UI from BeanShell, to which I've adding the bits which talk to the bash shell. It works ok on this Ubuntu machine, I've no idea what would be needed to set it up for a different OS. The source for that is here.

I started Scute, a desktop RDF toolkit, just over a year ago. I did get some bits working fairly well - I was using the SPARQL bits for real - but then I got distracted and left it largely unusable... This JEdwards bit of coding has got me back into it, and tightened up how I was thinking about the dev process. I must write this up properly. The main idea is, while it should be built from reusable components, the way it's setup as a whole will be optimized for how I want to work. Somewhat inspired by woodcarving, where a lot of the time what's best isn't a general purpose tool (wood router or software IDE) but a highly focused tool (1/4" No.4 fishtail gouge or JEdwards). If the resulting code is useful for other people, great, but the motivation isn't to create a product, just to help my own personal workflow. Horse before cart dogfood.

The reusable components part comes from testing. I'm lazy about tests at the best of times, and Scute is all about GUI so is a bit tricky to test. But I reckon component-level functional tests make a fair a substitute for unit tests. Anyhow, more about this another day.


danja
2012-01-18T19:12:14+01:00
scute terminal emulator jedwards sparql turtle syntax highlighter rdf
Related
Comments
Edit

Adding SPICE to the Semantic Web

Main Course

Here's a circuit:

distortion circuit
- and here's its SPICE model:

***
.INCLUDE la-components.mod

Rsrc 1 0 100E3
Rin 1 2 1E3
Rfeed1 2 3 10E3
Q1 3 0 4 BC109
Q2 3 0 4 BC179
Rfeed2 3 4 10E3
Xopamp 0 2 5 6 4 TL071
Rload 0 4 10E3
Vcc 5 0 15
Vee 6 0 -15

Vsrc 1 0 SIN(0V .1VPEAK 1KHZ)

.TRAN 10US 1000US
***

The .INCLUDE is as it sounds, the contents of that file are included in this model. After that it's describing a graph with two kinds of nodes: those associated with a component and connection nodes (i.e. common terminals/points/buses/PCB tracks...). Although the components kind-of contain arcs, they're hidden behind the component's connectors. The component's connectors are identified by their position in the space-separated data. On the schematic the nodes are marked in red.

Taking the first line:

Rsrc 1 0 100E3

This is interpreted via :

(a Resistor) <name> <node1connection> <node2connection> <value>

Rsrc is a 100k resistor connected between nodes 1 and 0

(Node/bus 0 is always ground)

Taking the first of the transistors:

Q1 3 0 4 BC109

a transistor of type BC109 called Q1 has its collector connected to node 3, base to node 0, emitter to node 4

The .TRAN line is used to run a simulation (a transient analysis), sampling every 10uS for 1000uS. I've not really figured out this side of things properly, couldn't get a straight .DC based transfer chart. But the sine wave will do for now.

Anyhow I can't go looking at a graph model for long without wondering how it could go on the Web. While there are no doubt loads of ways of doing it, the circuit definition can be transcribed into Turtle fairly directly. Bnodes could be used for the connection buses, but it's just as easy to name them. So making things up as I go along -

@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix spice: <http://purl.org/stuff/spice/> .
@prefix u: <http://purl.org/stuff/units/> .
@prefix d: <http://purl.org/stuff/devices/> .
@base <http://hyperdata.org/circuits/logamp/> .

<http://hyperdata.org/circuits/logamp> a spice:Circuit ;
dc:title "Log Amp" ;
dc:description "a modified log function amplifier" ;
spice:components ( <Rsrc> <Rin> <Rfeed1> ... <N0> <N1> ...) .

# Rsrc 1 0 100E3
<Rsrc> a spice:Resistor ;
rdfs:label "Rsrc" ;
spice:terminal1 <N1> ;
spice:terminal2 <N0> ;
u:ohms "100000" .

...

that seems ok, now for a transistor:

# Q1 3 0 4 BC109
<Q1> a spice:BJT ;
rdfs:label "Q1" ;
spice:terminal1 <N3> ;
spice:terminal2 <N0> ;
spice:terminal2 <N4> ;
spice:device d:BC109 .

that'll do.

Doing a .INCLUDE in general could really do with something from RDF core (ping RDF WG), but here it's providing other SPICE definitions of the components so it seems reasonable to be more explicit:

d:BC109 rdfs:isDefinedBy <http://hyperdata.org/circuits/logamp/components#BC109> .

which given that SPICE supports subcircuits (which is how TL071 is defined) provides a nice composition mechanism.

I reckon it should be straightforward to write a transformer from SPICE syntax to Turtle. Going the other way, the usual SPARQLing shouldn't be rocket science.

All seems doable. Homework. Rainy day.

Starter

I want to play with analog electronics again, stuff I used to do before the Web came along and ate up my cycles. My motivation now is mostly driven by the price of recording studio equipment. If, for example, I just want to invert the phase of a signal, I'd need to pay say $50+ for a passive DI box or $100+ for a pre-amp. This is a bit demoralising when the components are available for pennies (though hardware like connectors and cases can cost a lot more). Then of course there's the circuit hacking angle, it really is good fun. A project that's a permanent fixture in this space is the distortion pedal (like ghard's Big Muff) - the circuits aren't complicated, but getting a good sound is the Holy Grail, so this is what I'm going to play with first.

I did buy a bunch of components a while back, but haven't got much in the way of prototyping/test gear. A cheapo USB ADC will hopefully do for a makeshift oscilloscope for now, and I've just ordered the parts to put together a simple PSU (along with *lots* of oddments). But feeling a bit impatient, I thought I'd have a quick look what software was available these days for circuit simulation.

I don't know if I'm missing something, but things hardly seem to have progressed at all in the last couple of decades (but then again analog electronics hasn't really changed). The de facto standard is SPICE, and there are quite a few tools open source available for using it (ah, things weren't open source back in the day, that's progress). I won't bother linking to the individual bits, if you look for 'spice' in Synaptics a bunch show up, and they all seem to come under the umbrella of gEDA. Anyhow, after an hour or so's fiddling I was able to draw a little circuit using gschem, but I haven't yet managed to get it to generate a working netlist file (which specifies the inter-component connections for SPICE). I think I just need to sit down and check/add all the component attributes. But that's a bit tedious so I've just been playing with a SPICE file manually. Praise be to text formats.

The first problem here was finding simulation definitions of the components I want to use. The little circuit I want to test includes a common op-amp (TL071) and a pair of transistors, one NPN (BC 109), one PNP (BC 179). Took a lot of searching, and although (allegedly) many of the manufacturers do provide SPICE modules for their components, I eventually found what I needed on hobbiest sites. (Making the component module files doesn't look too difficult, it'd mostly mean copying values from a spec sheet into a SPICE definition - again, sounds tedious).

There is GUI adapter for running simulations, gspiceui (which I must have another look at now I've got a working model), but with the amount of trial and error I was having to do I settled back into the command-line tools. For future ref. it goes like this:

ngspice <filename>

This loads in the file and starts up an interactive shell. Took me a long time to figure out what to do next, but here are a couple of bits that worked for me. Once in the shell:

ngspice 1 -> run

Runs the simulation (the .TRANS bit). Then:

ngspice 2 -> plot V(1) V(4)

Produces a plot like this:

distorion plot

Red is the input (voltage on node 1), blue the output (voltage on node 4).

Certainly looks like distortion...wonder what it sounds like...

Pudding

Finding stuff and looking up references in this space is still fairly Paleolithic, so there's one application of exposing this kind of material as linked (wired!) data. But there are probably stacks of other more inspiring apps. Going totally blue sky, a globally distributed circuit could be rather cool. In the digital realm you for example could have a global computer that's built from just a few simulated gates on each of a million interconnected PCs. Bit like an extremely dumbed-down Web service/agent kind of thing.

In the analog realm it could get very wacky. Host your own local circuit subsystem, connect it to anyone else's. I guess you'd want to connect your inputs to other folks' outputs and offer outputs of your own. As long as you are limited to connecting your inputs to the rest of the world (or more versatile, I can only connect my output to your input if I have the appropriate rights) then subsystems should play nicely with each other. I see no reason why for control and audio signals you couldn't do this in real-time using existing streaming audio protocols (codec'ing locally to PCM for the instantaneous values).

This is pretty much what I assume some of the net-based recording systems that are around are doing. I must confess I've never looked into these, trying to mimic traditional recording/mixing stuff that way seems a bit of a non-starter because of the latency issues. But flipping it to a messier, slightly bonkers [insert pun about bipolar transistors] global analog synth kind of idea, then it starts to sound more fun.


danja
2011-02-23T22:00:24+01:00
spice turtle electronics semweb rdf
Related
Comments
Edit

del.icio.us bookmarks to RDF

The blogosphere seems to think Yahoo! is going to axe del.icio.us so I've knocked together a quick Python script to get my data out - 2317 occasionally annotated bookmarks. To use: make sure you've got Python first (!), download and install BeatifulSoup (navigate to the dir with setup.py, run python setup.py install), download the script and rename it to souper.py, get your del.icio.us bookmarks and rename to delicious.html. Then run python souper.py delicious.html > delicious.ttl and there you have the Turtle.

I've not checked the output particularly thoroughly, but I think it's ok (one shortcut I made was that any bookmarks that couldn't be converted to ASCII would get ignored). Here's my original bookmarks file and the same data in Turtle (7599 triples).

The Twitterati seem to be moving en masse to Pinboard, which has a sign-up fee of $7.42 but seems to have got good reviews.


danja
2010-12-17T00:24:50+01:00
script python turtle semweb rdf delicious tags
Related
Comments
Edit