Discussion:
How to use natdynlink
o***@pobox.com
2006-02-16 00:58:32 UTC
Permalink
Hello!

Included is the complete sample example of dynamically loading
a separately ocamlopt-compiled OCaml code. It is indeed quite involved,
probably more than needed (OTH, it was designed specifically for
MetaOCaml and I had no idea somebody would use it otherwise). The
files are committed into the MetaOCaml repository; if Walid wants, the
example can become part of the next MetaOCaml distribution. I think we
once toyed with an idea of writing a paper (for the previous MetaOCaml
workshop), but it all fizzled.

To publicly answer the question about unloading of the loaded
code: it is not currently possible. The loaded code stays forever, for
really good reasons. This can be fixed, but it takes a lot of work and
requires really compelling reason and strong motivation. Incidentally,
Haskell-plugins likewise stay loaded forever and can't be
unloaded. So, we're in the same boat.


# This is a shell archive. Save it in a file, remove anything before
# this line, and then unpack it by entering "sh file". Note, it may
# create directories; files and directories will be owned by you and
# have default permissions.
#
# This archive contains:
#
# test-main.ml
# test_int.ml
# test_int.mli
# test_sub.ml
# Makefile
#
echo x - test-main.ml
sed 's/^X//' >test-main.ml << 'END-of-test-main.ml'
X(* An example of using natdynlink
X This is the main code, that will dynamically load natively
X compiled file test-sub.
X We assume for this example that test-sub returns two pieces
X of data: an integer and a function (closure) of the signature
X int->string. See test-int.mli for details
X*)
X
Xopen Test_int
X
Xlet () = print_endline "test-main: starting"
X
X
X(* The first string argument of Natdynlink.loadfile is the name of
X the .so file.
X If the name has no directory component, this .so file
X will be looked up as a regular dynamically loaded library:
X with the help of ld.config, LD_LIBRARY_PATH, etc.
X Often it's less hassle just to specify the directory path,
X even the dummy one as "./", and avoid the LD_LIBRARY_PATH hassles.
X
X The second string argument is the entry point:
X let entry_point = "caml" ^ modulename ^ "__entry"
X where the modulename is the name of the Ocaml module in
X test_sub (which is Test_sub).
X*)
X
Xlet _ = let modulename = "Test_sub" in
X let entry_point = "caml" ^ modulename ^ "__entry" in
X Natdynlink.loadfile "./test_sub.so" entry_point
X
Xlet () = print_endline "test-main: loaded test_sub"
Xlet () = Printf.printf "Result from test_sub: %d and the function yields: %s\n"
X !sub_result1 (!sub_result2 42)
X
Xlet () = print_endline "test-main: done"
END-of-test-main.ml
echo x - test_int.ml
sed 's/^X//' >test_int.ml << 'END-of-test_int.ml'
X(* The interface between the dynamically loaded module test_sub.ml and
X the main module test-main.ml
X The interface is in the form of reference cells. The dynamically loaded
X module will store the real values in them up when loaded.
X This module merely initializes the interface values.
X*)
X
Xlet sub_result1 = ref 0
Xlet sub_result2 = ref (fun x -> failwith "not set yet")
X
END-of-test_int.ml
echo x - test_int.mli
sed 's/^X//' >test_int.mli << 'END-of-test_int.mli'
X(* The interface between the dynamically loaded module test_sub.ml and
X the main module test-main.ml
X The interface is in the form of reference cells. The dynamically loaded
X module will store the real values in them up when loaded.
X*)
X
Xval sub_result1 : int ref
Xval sub_result2 : (int->string) ref
X
END-of-test_int.mli
echo x - test_sub.ml
sed 's/^X//' >test_sub.ml << 'END-of-test_sub.ml'
X(* This is the module being dynamically loaded by the test-main
X The name of this module is Test_sub (created from the file name
X in the traditional OCaml way
X*)
X
Xopen Test_int
X
Xlet myvalue = 42 + 42
Xlet myfun x = string_of_int x ^ " and " ^ string_of_int (2*myvalue)
X
X(* Now set up the return values, in the Test_int interface *)
X
Xlet () = sub_result1 := myvalue
Xlet () = sub_result2 := myfun
END-of-test_sub.ml
echo x - Makefile
sed 's/^X//' >Makefile << 'END-of-Makefile'
Xinclude ../config/Makefile
XCC=$(NATIVECC)
X
XLIB=libnatdyn.a
X
Xclean:
X rm -f *.o *.s $(LIB) *.cmi *.cmx test_sub.so
X
X
X
X# Running the sample test
X
XCAMLOPT=$(BINDIR)/ocamlopt
XCAMLFLAGS=-verbose -S -ccopt -Wl,-E
XCOMPFLAGS=-S -dparsetree -drawlambda -dlambda
X
X.SUFFIXES: .ml .mli .cmx .cmi
X
X.mli.cmi:
X $(CAMLOPT) -c $(COMPFLAGS) $<
X
X.ml.cmx:
X $(CAMLOPT) -c $(COMPFLAGS) $<
X.ml.o:
X $(CAMLOPT) -c $(COMPFLAGS) $<
X
X# Here, often MKSHAREDLIB=gcc -shared -o
Xtest_sub.so: test_int.cmi test_sub.o
X $(MKSHAREDLIB) $@ -Wl,-E test_sub.o
X
Xtest-main: $(LIB) test-main.ml \
X test_sub.so \
X natdynlink.cmi natdynlink.cmx \
X test_int.cmi test_int.cmx
X $(CAMLOPT) $(CAMLFLAGS) -o $@ natdynlink.cmx test_int.cmx \
X test-main.ml $(LIB)
X ./$@
END-of-Makefile
exit
Walid Taha
2006-02-16 01:16:42 UTC
Permalink
Hi Oleg,

Thank you very much for the detailed responce. I think it would be fine
to include this in the MetaOCaml distribution, and it certainly looks like
it can develop a life of its own.

When you add the example, it would be very useful to include
documentation. It would be great if Alessandro and malc get involved in
this, at least in as far as saying "Yes, it looks good now". Naturally,
more involvment would be better.

Please remember to sign the documentation and the various files (this is
for Oleg and for everyone that gets involved).

Walid.
Alessandro Baretta
2006-02-16 09:01:09 UTC
Permalink
Post by o***@pobox.com
Hello!
Included is the complete sample example of dynamically loading
a separately ocamlopt-compiled OCaml code. It is indeed quite involved,
probably more than needed ...
What a man, Oleg! What a man!

I love the Jedi programming paradigm: "Use the source, Luke!" I have read the
source, the Makefile in particular, and found most of the information I need.
The other bit of information I needed--the meaning entry parameter to
loadfile--I am able to infer from the following line in test-main.ml.
Post by o***@pobox.com
let entry_point = "caml" ^ modulename ^ "__entry"
It seems to me that the appropriate way to integrate Natdynlink with the
AS/Xcaml is to provide an alternative implementation of the Dynlink signature,
based on Natdynlink, so that I do not need to use any horrendous camlp4 hacks in
the application source code. Findlib provides the machinery needed to select the
appropriate dynlink library depending on the active predicates.

Gentlemen, we are very close to having a full-fledged dynamic web application
server for Ocaml programs. Stay tuned.

Alex
--
*********************************************************************

Ing. Alessandro Baretta

Studio Baretta
http://studio.baretta.com/

Consulenza Tecnologica e Ingegneria Industriale
Technological Consulting and Industrial Engineering

tel. +39 02 370 111 55
fax. +39 02 370 111 54
Alessandro Baretta
2006-02-16 11:15:21 UTC
Permalink
Post by Alessandro Baretta
Post by o***@pobox.com
Hello!
Included is the complete sample example of dynamically loading
a separately ocamlopt-compiled OCaml code. It is indeed quite involved,
probably more than needed ...
What a man, Oleg! What a man!
Let me thank Oleg once again. The AS/Xcaml now correctly supports Natdynlink. My
trouble now is that I need to hack support for natdynlinking findlib-managed
libraries. I would expect this to be a minor issue, though.

Alex
--
*********************************************************************

Ing. Alessandro Baretta

Studio Baretta
http://studio.baretta.com/

Consulenza Tecnologica e Ingegneria Industriale
Technological Consulting and Industrial Engineering

tel. +39 02 370 111 55
fax. +39 02 370 111 54
Loading...