Marionnet invoking

Hello, and thanks a lot for your interest.

Your announcement was a very pleasant surprise for us: we have always thought that having pre-compiled packages was a priority for inexperienced users (who are certainly part of our main target), but that making packages was a lot of work. We had debian packages contributed by Jonathan Roudiere for an old version, and Franck Butelle has generated more recent Ubuntu/debian (and RPM) packages; anyway the mechanism for automatically generating packages is currently *not* integrated in the main repositories, which entails that we (Jean-Vincent and I) are *not* able to easily re-generate packages for new versions. We would like to improve the situation, and we could definitely use help in that area. Would you consider contributing your scripts for building packages? We would like to merge the what we have in order to reduce the maintenance burden and ease the release process in the future, so that a "make deb" suffices to create and possibly upload packages with the right name.

The message "No bytecode file specified." sounds a little strange (invoking ocamlrun with no parameters?), and neither I nor Jean-Vincent have ever seen it when using Marionnet. Anyway, that's not terribly important: actually, packaging native binaries only and ignoring bytecode would be perfectly fine. Each program, marionnet and marionnet-daemon, is compiled in two different fashions: a slower but very portable bytecode file, and a fast native only available on some architectures -- there is no other difference. Now, performance is not much of a concern for Marionnet, as the part written in OCaml is not a CPU bottleneck; and portability isn't a big deal either, since UML only works on x86 and x86_64 anyway, and both platforms are also supported by the native OCaml compiler. So we can use bytecode files just internally for testing (bytecode files can be *compiled* faster), and avoid packaging them.

In the ideal case we would like you to help maintain the packaging subsystem in the future: it would be, we think, a very light task after the system is written, but the occasional fix may be needed once in a while when something breaks. Of course in that case we would be more than happy to add your name to our web site, and make you and official part of the team -- unless you have something against the idea.

Of course we would love a Slovak translation! You can use the Launchpad web interface at https://translations.launchpad.net/marionnet . If you have problems with that or don't like it, please ask again and we will suggest an alternative method involving a text file. As soon as your Slovak translation becomes reasonably complete, we can enable the language for the next release (it's just about changing the file po/LINGUAS).

Good catch Franck, thanks: stripping bytecode files breaks them when they also contain compiled C code. Yes, I agree it's counter-intuitive.
Slavko, did you strip bytecode files before installing them? This would explain the issue.

Hi,

i can share my packaging (maybe is needed exatly to write - yes, i can contribute), of course, all sources are here:

http://debs.slavino.sk/pool/main/m/marionnet
http://debs.slavino.sk/pool/main/m/marionnet-kernels

exact links to marionnet package code is here:
http://debs.slavino.sk/pool/main/m/marionnet/marionnet_0.90.6-3.debian.tar.gz

the marionnet-kernels package i have created as native debian package, so it have no debian.tar.gz, but all is in:
http://debs.slavino.sk/pool/main/m/marionnet-kernels/marionnet-kernels_0.90.6.tar.gz

Be free to choice linking to my repo, or copy to your repo. But now the packages are not perfect. I am able to create package for virtual filesystems too, but i cannot upload it to my repo, because it is too large :-)

Franck, you are right, binaries was stripped (auto by debhelper), i will preffer include only *.native binary versions in packages, as suggested solution by Luca. Perhaps i will able create separate binary packages (for daemon and GUI), one with bytecode (not stripped) and one with native files (stripped), later. But i don't know if is this needed.

The packaging your software was not terrible, could you do perfect job. Only some changes was needed. In Makefile and Makefile.local i added $(DESTDIR), to all install targets and changed prefix from /usr/local to /usr. Both changes are as patch in mentioned *.debian.tar.gz. After these changes, the packages are automatically generated by debhelper, with minimal rules file.

I prefer do not use the web interface for translating (i am translating more projects). I patched the LINGUAS file, and initalized the sk.po file from messages.pot. Now i have translated only some small strings (cca 25%), to see if it works. My question was only to where i need to send full translated file.

Now i go to play with removing bytecode, adding menu and desktop files, etc.Full translating will be the job, after packaging will be completed. For now i know about another problem in build dependencies, while no pdf documentation file is generated in pbuilder chroot, but i do not know what is missing...

I will post info here after success. Or is here better way to communicate?

Thanks Franck Butelle, that solved my question.

i forgot links to ocamlbicks package:

http://debs.slavino.sk/pool/main/m/marionnet-ocamlbricks
http://debs.slavino.sk/pool/main/m/marionnet-ocamlbricks/marionnet-ocamlbricks_0.90.3-1.debian.tar.gz

:-)

Hello again Slavko, and thanks. Your contribution is really great news.

As for sending us the sk.po file, well, that's easy; you can mail me, Jean-Vincent or Franck. In order to communicate more effectively I've also taken the liberty to add you to the marionnet-dev team; can you please also subscribe to the mailing list? I can't do it myself. Subscribing is important if you want to communicate with us, since launchpad lists silently ignore messages from non-members. (Yes, I know... I already complained, but the Launchpad people seem to prefer simplicity to usability).
The subscription link is near the end of:
https://launchpad.net/~marionnet-dev

I agree that bytecode packages are not actually needed, since on all supported architectures we have native executables as well, and they are faster...

Working on desktop menus and launcher desktop files is also something we have wanted to have well-supported directly in our source tree for a long time. That's very helpful, thanks. Please don't hesitate to write us for any question.

I don't know about pbuilder. (Does Franck know, by chance?); if what you mean by the PDF documentation is the Texinfo manual in doc-src/, then that's not prioritary: that documentation is not very up-to-date, and we need to update it to reflect the recent major changes in Marionnet.

Best regards,

Slavko, I've also added your Jabber contact.

maillist subscription - done
creating desktop file - done
removing bytecode files - done
ignoring missing pdf - done

the new version are uploaded, now tested with daemon too and all seems to work. But, some testing by someone other than I will be welcomed! By uploading new version, all links above which points to debian.tar.gz are broken, but links to directories must works.

pbuilder is debian tool by which is package compilation/creation always doned in clean chroot environment. Only defined build dependencies (in package control file) are downloaded and instaled before compiling and building package (and removed after build). By this, the package building is not touched by my real system (with a lot of different software installed) and i am able to build packages for 32b and 64b architectures. For my problem with PDF this means, that i have missing some dependencies, while on my real system is PDF generated, but in clean system not.

next communication will be done by jabber or email.

regards

Hi,

I was able to rebuild marionnet-ocamlbricks
http://marionnet.dachary.org/packaging-farm/marionnet-ocamlbricks/gnulinux/debian/x86_64/wheezy/src/

but it failed on marionnet with:

Just for debugging: PP_OPTION is "camlp4of -I /usr/lib/ocaml/ocamlbricks gettext_extract_pot_p4.cmo option_extract_p4.cmo raise_p4.cmo log_module_loading_p4.cmo -I chip"
Building meta.ml...
Success.
Manually pre-copying "gettext_extract_pot_p4.conf"...
Manually pre-copying "chip/chip_parser_p4.ml"...
Manually pre-copying "scripts/can-directory-host-sparse-files.sh"...
Manually pre-building "chip/chip_parser_p4.cmo"...
File "chip/chip_parser_p4.ml", line 52, characters 69-72:
While expanding quotation "ctyp" in a position of "expr":
  Parse error: EOI expected after [quotation of type] (in [quotation of type])

File "chip/chip_parser_p4.ml", line 1, characters 0-1:
Error: Preprocessor error
make[6]: *** [_build/chip/chip_parser_p4.cmo] Error 2
make[5]: *** [manually_pre_actions] Error 1
dh_auto_build: make -j1 returned exit code 2
make[4]: *** [override_dh_auto_build] Error 2
make[3]: *** [build] Error 2

the full lot can be found at
http://marionnet.dachary.org/packaging-farm/marionnet/build/wheezy-x86_64.out

I would appreciate your input on how to fix it.

Cheers

The last release of marionnet must be compiled with OCaml 3.11, not with the new version (3.12). This last version of ocaml give us some serious problems at run-time, and not just the problem that you have encountered at compile-time, which is easy to fix (and which was already fixed in the trunk).

The porting of marionnet (and ocamlbricks) to OCaml 3.12 is a high priority task for us. We expect to fix the problems in the next weeks, in order to be able to release the version 0.90.7.

I reverted to squeeze because it has 3.11 ( http://qa.debian.org/developer.php?packages=ocaml ) and it works better, indeed.

Both marionnet-ocamlbricks
http://marionnet.dachary.org/packaging-farm/marionnet-ocamlbricks/gnulinux/debian/x86_64/squeeze/src/
and marionnet
http://marionnet.dachary.org/packaging-farm/marionnet/gnulinux/debian/x86_64/squeeze/src/
were successfully built.

I then tried to create marionnet-kernels but got the following error:

dpkg-shlibdeps: error: couldn't find library libutil.so.1 needed by debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian (ELF format: 'elf32-i386'; RPATH: '/lib').
Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/marionnet-kernels.substvars debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-ghost returned exit code 2
make[3]: *** [binary] Error 9

the full log is at
http://marionnet.dachary.org/packaging-farm/marionnet-kernels/build/squeeze-x86_64.out

It may be because this package is created in to amd64 / x86_64 environment and that the debian package somehow assumes a i386 architecture. I did not examine the source of the problem yet and I would appreciate any advice you may have.

Cheers

Bonjour Loic,

Je commute en français, ça ira bcp plus vite.
Ni Luca, ni moi nous sommes en mesure de t'aider sur ce problème de génération
de paquets. En revanche, on peut te donner un fichier qui répond probablement à
plusieurs questions, dont peut-être celle que tu pose. Il s'agit du travail
qu'avait réalisé Jonathan Roudière (embauché en 2009 sur des fonds du projet
avant de disparaitre vers d'autres horizons). Tout est dans ce fichier :

http://www.marionnet.org/download/ghost-kernel-2.0/ghost-kernel-2.0.tar.bz2

Il s'agit essentiellement de :

1) une mise-à-jour du patch noyau (réalisé au départ par Luca pour le 2.6.18)
=> sous-répertoire kernel-patch/
voir par exemple : kernel-patch/2.6.32/

2) une mise-à-jour des utilitaires "ghostify" (rendre une interface réseau
invisible à l'utilisateur) et "unghostify" (lui redonner visibilité) écrites au
départ par Luca
=> sous-répertoire user-tool/ghost-2.0/

3) (ce qui t'intéresse le plus) les règles de construction de ces deux paquets
=> fichier debian/rule
où on peut lire :
..
PACKAGE_MARIONNET=marionnet-kernel-$(DEB_KERNEL_VERSION)
PACKAGE_GHOST2=ghost2

On y trouve aussi :
PACKAGE_UML=uml-kernel-$(DEB_KERNEL_VERSION)-ghost
qui ne devrait pas t'intéresser (ça doit être la même chose que
PACKAGE_MARIONNET avec un .config en moins).

Remarque au passage : le paquet ghost2 (ghostify/unghostify) a vocation à être
installé sur le système de fichiers d'une machine virtuelle (il sera donc utile
seulement pour les machines virtuelles debian). C'est une exception, le reste
des paquets doit toujours être installé sur la machine hôte hébergeant le réseau
virtuel.

Nous aurions du faire depuis longtemps le merge entre ce travail et celui de
slavko.

J'espère que ça pourra te faire avancer...
Jean-Vincent

Est-ce que marionnet-kernel est un package optionel ? Autrement dit, est-ce que marionnet peut fonctionner sans lui sans dégradation génante ?

Hi,

Dňa Mon, 24 Oct 2011 13:45:40 -0000 Jean-Vincent Loddo
<email address hidden> napísal:

> Nous aurions du faire depuis longtemps le merge entre ce travail et
> celui de slavko.

i speek only in slovak, czech, russian and english...

regards
--
Slavko
http://slavino.sk

Loic is investigating the possibility of applying his tool
http://packaging-farm.dachary.org/trac/
to the marionnet project, starting from your work (thanks again!) and, maybe,
from the work of Jonathan Roudière. We was speaking French for simplicity,
because the topic seemed not very interesting to us for final users.

> Nous aurions du faire depuis longtemps le merge entre ce travail et
> celui de slavko.
This sentence expresses our regret about the non-integration of your work and
that of Jonathan in the project, essentially for our lack of time and
competences in packaging.

> Loic Dachary posted a new comment:
> Est-ce que marionnet-kernel est un package optionel ? Autrement dit,
> est-ce que marionnet peut fonctionner sans lui sans dégradation génante ?

It's very hard to consider marionnet-kernel as optional because without it,
marionnet (in the sense of the "pilot" of the virtual network) will not be able
to launch any virtual machine...
In order to launch a VM, the pilot must have at least an UML kernel (with our
patch) and at least a filesytem image available. In other terms, the pilot needs
at least a couple (kernel, filesystem-image) installed in the host.

Cheers

Ahoj,

Dňa Tue, 25 Oct 2011 08:55:47 -0000 Jean-Vincent Loddo
<email address hidden> napísal:

> Your question #151530 on marionnet changed:
> https://answers.launchpad.net/marionnet/+question/151530
>
> to the marionnet project, starting from your work (thanks again!) and,
> maybe, from the work of Jonathan Roudière. We was speaking French for

I understand of using the French. I asked because my name was mentioned
only and i wasn't able to response. But now i see, that my response is not
needed.

> > Nous aurions du faire depuis longtemps le merge entre ce travail et
> > celui de slavko.
> This sentence expresses our regret about the non-integration of your
> work and that of Jonathan in the project, essentially for our lack of
> time and competences in packaging.

thanks for translation :-)

regards

--
Slavko
http://slavino.sk

Hi Slavko,

Sorry for the french speaking escapade ;-)

Regarding marionnet-kernel I now understand it is an essential part of marionnet, thank to Jean-Vincent Loddo explanations. Although it would be good to upgrade, I would be happy to just re-generate the packages you built. It is easier for me to start with a consistent base and then upgrade or fix what is wrong. It helps me understand what I am dealing with.

I would very much appreciate if you have a hint about the problem I encountered (cited below for your convenience). Have you ever seen this error ? Do you have a suggestion about where it comes from ?

Thanks for your help and thanks for packaging marionnet in the first place.

----
I tried to create marionnet-kernels but got the following error:

dpkg-shlibdeps: error: couldn't find library libutil.so.1 needed by debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian (ELF format: 'elf32-i386'; RPATH: '/lib').
Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/marionnet-kernels.substvars debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-ghost returned exit code 2
make[3]: *** [binary] Error 9

the full log is at
http://marionnet.dachary.org/packaging-farm/marionnet-kernels/build/squeeze-x86_64.out

Ahoj,

Dňa Wed, 26 Oct 2011 20:55:53 -0000 Loic Dachary
<email address hidden> napísal:

> Sorry for the french speaking escapade ;-)

No problem here ;-)

> I would very much appreciate if you have a hint about the problem I
> encountered (cited below for your convenience). Have you ever seen this
> error ? Do you have a suggestion about where it comes from ?

I was not tried to create source package for kernels yet. For me i
created only binary package, which contains precompiled kernels,
downloaded from marionnet's download page.

My initial ida was do not provide compiled package for kernels, but
package, which downloads precompiled kernels from marionnet's home, by
similar manner, as package for flashplayer plugin does his own job. This
was discused here, but i am not sure now, if this discussion was some
end... This idea was initialized while i have no enough space (and
traffic) on my hosting, to store and provide big packages in my repo.

> ----
> I tried to create marionnet-kernels but got the following error:
>
> dpkg-shlibdeps: error: couldn't find library libutil.so.1 needed by
> debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian
> (ELF format: 'elf32-i386'; RPATH: '/lib'). Note: libraries are not
> searched in other binary packages that do not have any shlibs or symbols
> file. To help dpkg-shlibdeps find private libraries, you might need to
> set LD_LIBRARY_PATH. dh_shlibdeps: dpkg-shlibdeps
> -Tdebian/marionnet-kernels.substvars
> debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-debian
> debian/marionnet-kernels/usr/lib/marionnet/kernels/linux-2.6.18-ghost
> returned exit code 2 make[3]: *** [binary] Error 9
>
> the full log is at
> http://marionnet.dachary.org/packaging-farm/marionnet-kernels/build/squeeze-x86_64.out
>

I am not sure, why this happens, but perhaps amd64 - i386 issue? Consider
this part of the error message:

(ELF format: 'elf32-i386'; RPATH: '/lib').
              ^^^^^^^^^^

On amd64 system try to install the libc6-i386 package and precise
package's dependencies for it.

Another reason, which comes to my mind, is debian's multiarch (new in
wheezy), by which some .la files contains unproper paths, read more here
http://wiki.debian.org/ReleaseGoals/LAFileRemoval

(but when i encounter with this, the error message was a bit different)

regards

--
Slavko
http://slavino.sk

Hi,

Thank you for the explanation. The fact that only binary packages are generated explains a lot :-) Your approach is completely understandable: creating custom kernels is a little tricky.

Unfortunately the tool used to create the packages ( packaging-farm ) will not help resolve packaging issues, it is meant for already existing packages. As soon as they are available, it could be applied fairly easily. The virtual machine will be kept alive for when this time arrives ;-)

Cheers