EzDevInfo.com

Logging xm console output in xen?

Is there a way I can log the output of xm console in to some file?

Can I set something in the domain configuration file which automatically does this for me when I issue xm create?

I tried:

xm console | tee domU-console.log

... , but that does not fit in my case. I want to log the output asynchronously in background.

CUDA / OpenCL within a Virtual Machine / Hypervisor

Anyone know of any virtualisation solutions that either allow CUDA/OpenCL access directly or implement CUDA/OpenCL abstraction.

UPDATE: Thank you those who commented. While classical 'desktop' virtualization would be nice, I suspect the likes of Xen would be closer to the mark.

Detect when application is running in a VM environment

Looking for ideas on how to detect when our Windows application is running in one of the following VM environments. Some starter ideas for detection are in parentheses. There may be (much) better detection techniques - the starter ideas I've come up with are based on my Google research.

• VMWare (looking for the presence of optional VMware Tools is one way)

• Microsoft Virtual PC (have a device named "Virtual HD" for their IDE disks, "MS Virtual SCSI Disk Device" for their SCSI disks)

• Citrix Xen

• Sun Virtual Box

Thank you, Malcolm

Why are many "bare kernel+app" solutions targetting xen?

There's a number of projects that produce complete images from your app, bootable on xen as a machine. For example Erlang on Xen, OpenMirage, HaLVM, and others.

Why is Xen the default hypervisor for them? Does it provide some interface that makes these projects easier (as opposed to KVM, VmWare, etc.), or is it just the project developers' choice?

Xen Server Cluster [closed]

I have upwards of 30 Dell GX2xx models doing nothing. I've decided on using them to build a cluster, but I am lost as far as getting started. I've used ClusterKnoppix, and even straight OpenMosix in the past, but those projects are, very sadly, dead now.

I've checked out Xen, to an extent. I don't know if Xen is the solution I need. I'd like to have the ability to spin up a few VMs (when I need) in a server pool, with the VMs having the ability to run just off of resources in the pool, leaving me without the care of which node they run on.

I need some insight here... Thanks all!

Detect virtualized OS from an application?

I need to detect whether my application is running within a virtualized OS instance or not.

I've found an article with some useful information on the topic. The same article appears in multiple places, I'm unsure of the original source. VMware implements a particular invalid x86 instruction to return information about itself, while VirtualPC uses a magic number and I/O port with an IN instruction.

This is workable, but appears to be undocumented behavior in both cases. I suppose a future release of VMWare or VirtualPC might change the mechanism. Is there a better way? Is there a supported mechanism for either product?

Similarly, is there a way to detect Xen or VirtualBox?

I'm not concerned about cases where the platform is deliberately trying to hide itself. For example, honeypots use virtualization but sometimes obscure the mechanisms that malware would use to detect it. I don't care that my app would think it is not virtualized in these honeypots, I'm just looking for a "best effort" solution.

The application is mostly Java, though I'm expecting to use native code plus JNI for this particular function. Windows XP/Vista support is most important, though the mechanisms described in the referenced article are generic features of x86 and don't rely on any particular OS facility.

Which Linux distribution should I use as a Xen host? [closed]

I ordered a server for the home office and I would like to partition it with Xen. I think this will keep things clean and easier to maintain. I will be running things like MySQL, PostgreSQL, Tomcat, and my own code.

What freely available Linux distribution has the best Xen hosting facilities?

triple-dash on linux kernel command line switches

I'm trying to add a switch to send linux kernel log to the serial console on XenServer6.

The kernel command options can be edited on the EXTLinux config file ( /boot/extlinux.conf ).

Here is an excerpt:

serial 1 115200
default xe
prompt 1
timeout 50

label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz mem=1024G dom0_max_vcpus=4 dom0_mem=752M lowmem_emergency_pool=1M crashkernel=64M@32M console= vga=mode-0x0311 --- /boot/vmlinuz-2.6-xen root=LABEL=root-tfnnfzfp ro xencons=hvc com2=115200,8n1 console=com2 console=hvc0 console=tty0 quiet vga=785 splash --- /boot/initrd-2.6-xen.img

label xe-serial
  # XenServer (Serial)
  kernel mboot.c32
  append /boot/xen.gz com1=115200,8n1 console=com1,vga mem=1024G dom0_max_vcpus=4 dom0_mem=752M lowmem_emergency_pool=1M crashkernel=64M@32M --- /boot/vmlinuz-2.6-xen root=LABEL=root-tfnnfzfp ro console=tty0 xencons=hvc console=hvc0 --- /boot/initrd-2.6-xen.img

What is the meaning of the triple dashes ( --- )on the command line?

Is it loading 3 boot files?

The state of XEN vs KVM for inhouse virtual server farm like live farm

so we have a life server Farm, conventional stuff not virtualized. This is to stay that way for the time beeing. The servers we are talking about are: Loadbalancer, DB master, DB Slaves (2x), Webservers (2x), CMS server. All in all 7 servers per farm.

We want to have one to many rather exact "copies" of the life structure for inhouse purposes.

• Testing/Staging: to find bugs before live that might slip through if we do not use the same software-configuration
• Debugging/Benchmarking: Same like testing but with additional tools. debug flags and isolated from Tetsing/Staging to not interfeer with QA. Intended to get a deeper insight into how things are on our live-systems
• Development/Experimental: Changing System Components, Software-Versions, Libraries and Configuration to improve performance, developer effectivness, future-proofing our systems etc.

All in all 3 virtual "farms" a 7 virtual servers each, neatly isolated.

Now I know this can be achieved with many different flavors of virtualization. The Questions is, what is the best? VMware not beeing open source is nothing we would like to use on a strategic point of view. Looking at XEN and KVM, the two biggest players in open source virtualization I would love to hear some advise on what/how to select. The Web seems undecided.

Additional Info:

• "life time" of the server hardware that we purchase will be 2-3 years
• There is the possibility that we might migrate systems into cloud-environments later, if that is a factor to consider in the selection of the virtualization-technology
• Isolation of Server-Farms/Servers in High-Load scenarios is important. QA should not suffer when someone wrecks havoc in the experimental farm
• Efficent ressource usage is of course appreciated (memory overcommitment/shared pages? Automaticly Shared objects on hard-drive like the linux-vserver hashify?)
• Regular maintenance requierements and easy of management

I know things are in the flow but I would appreciate if you would tell me your opinion on what to choose right now seeing we want to live with the technique for at least 3 years, build up and reuse know-how. Also maybe in the end we can all realize that neither one, XEN or KVM is deciding the battle but other factors. Enlightenment in that regards is even more valuable to us right now.

p.s.: and lets not flame :D

How to build a Xen unikernel of Mirage OS

I read this document(http://openmirage.org/wiki/hello-world) and tried to build a xen unikernel as follows:

mirage configure --xen
make depend
make    

But failed at compilation:

ocamlbuild -classic-display -use-ocamlfind -pkgs lwt.syntax,mirage-console.xen,mirage-types.lwt -tags "syntax(camlp4o),annot,bin_annot,strict_sequence,principal" -tag-line "<static*.*>: -syntax(camlp4o)" -cflag -g -lflags -g,-linkpkg,-dontlink,unix main.native.o
ocamlfind ocamldep -package mirage-types.lwt -package mirage-console.xen -package lwt.syntax -syntax camlp4o -modules main.ml > main.ml.depends
ocamlfind ocamlc -c -g -annot -bin-annot -principal -strict-sequence -package mirage-types.lwt -package mirage-console.xen -package lwt.syntax -syntax camlp4o -o main.cmo main.ml
ocamlfind ocamlopt -c -g -annot -bin-annot -principal -strict-sequence -package mirage-types.lwt -package mirage-console.xen -package lwt.syntax -syntax camlp4o -o main.cmx main.ml
pkg-config --print-errors --exists mirage-xen
Package mirage-xen was not found in the pkg-config search path.
Perhaps you should add the directory containing `mirage-xen.pc'

to the PKG_CONFIG_PATH environment variable No package 'mirage-xen' found make: *** [build] Error 1

I installed mirage-xen by opam install mirage-xen successfully. But in my .opam directory, there are no mirage-xen.pc.

Improving JVM performance in a virtualized environment

What are the points to consider in improving JVM performance in a virtualized environment (Vmware/XenServer)?

• Whare are the settings to improve JVM in a virtualized environement?
• Is there any JVM specially made for virtualized environment?
• Or is there a JVM that is used by a Java client and send bytecodes to a server and execute it from there?

How to Extract some Fields from Real Time Output of a Command in Bash script

I want to extract some fields out of output of command xentop. It's like top command; provides an ongoing look at cpu usage,memory usage,...in real time. If I run this command in batch mode, I will have its output as you see in a file:

      NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_RD   VBD_WR  VBD_RSECT  VBD_WSECT SSID
  Domain-0 -----r      13700   33.0    7127040   85.9   no limit       n/a     8    0        0        0    0        0        0        0          0          0    0
     fed18 -----r        738  190.6    1052640   12.7    1052672      12.7     3    1   259919     8265    1        0    82432    22750    2740966    1071672    0

and running this

cat file| tr '\r' '\n' | sed 's/[0-9][;][0-9][0-9][a-Z]/ /g' | col -bx | awk '{print $1,$4,$6}'

on this file gives me what I want

NAME CPU(%) MEM(%)
Domain-0 33.0 85.9
fed18 190.6 12.7

but my script doesn't work on realtime output of xentop. I even tried to just run xentop one time by setting itteration option as 1(xentop -i 1) but It does not work! How can I pipe output of xentop as "not" realtime to my script?

Problem executing script using Python and subprocces.call yet works in Bash

For the first time, I am asking a little bit of help over here as I am more of a ServerFault person.

I am doing some scripting in Python and I've been loving the language so far yet I have this little problem which is keeping my script from working.

Here is the code line in question :

subprocess.call('xen-create-image --hostname '+nom+' --memory '+memory+' --partitions=/root/scripts/part.tmp --ip '+ip+' --netmask '+netmask+' --gateway '+gateway+' --passwd',shell=True)

I have tried the same thing with os.popen. All the variables are correctly set.

When I execute the command in question in my regular Linux shell, it works perfectly fine but when I execute it using my Python scripts, I get bizarre errors. I even replaced subprocess.call() by the print function to make sure I am using the exact output of the command.

I went looking into environment variables of my shell but they are pretty much the same... I'll post the error I am getting but I'm not sure it's relevant to my problem.

Use of uninitialized value $lines[0] in substitution (s///) at /usr/share/perl5/Config/IniFiles.pm line 614. Use of uninitialized value $_ in pattern match (m//) at /usr/share/perl5/Config/IniFiles.pm line 628.

I am not a Python expert so I'm most likely missing something here.

Thank you in advance for your help,

Antoine

EDIT

Following miax's advice, I stopped using shell=True. Instead I took a look at the Python documentation for subprocess and used the following piece of code :

cmd = 'xen-create-image --hostname '+nom+' --memory '+memory+' --partitions=/root/scripts/part.tmp --ip '+ip+' --netmask '+netmask+' --gateway '+gateway+' --passwd'
args = shlex.split(cmd)
subprocess.call(args)

Sadly, it doesn't change anything...

EDIT2

I have used the tip given by miax but I still get the above error... Here is the code that I have used.

cmd = ['xen-create-image', '--hostname', nom, '--memory', memory, '--partitions=/root/scripts/part.tmp', '--ip', ip, '--netmask', netmask, '--gateway', gateway, '--passwd']
subprocess.call(cmd)

This is really strange... The exact command works fine when I run it in the regular shell...

How can I get OProfile / Xenoprof to use event counters when running on EC2?

I'm writing a cluster-wide profiling tool for EC2 that is designed to collect data by running OProfile on each instance. I'd eventually like to aggregate the XML output generated by opreport, and analyze it later.

Right now, I'm stuck on getting OProfile to see hardware events -- all I can do is build profiles in Timer Interrupt mode. I have a feeling this is due to the fact that EC2 instances run in a virtualized environment, where OProfile cannot see hardware events.

I looked into this hunch a bit, and came up with Xenoprof as a possible solution, but there's very limited documentation on how to get it up and running with EC2.

If anyone's had any success running Xenoprof on EC2 (and I know there must be some successes, because there are several papers about it), could you clue me in as to what's going wrong?

When I run yum install oprofile

and then man opcontrol

on the instance, I see the options for xen are there, which I think means that Xenoprof must be installed. I just don't know what to do to get the hardware events as my samples.

Any advice would be much appreciated! Thanks.

How is Docker related to exokernal approach like Mirage OS?

How is Docker related to exokernal approaches like MirageOS?

For some information about MirageOS see:

• http://www.xenproject.org/developers/teams/mirage-os.html
• http://www.se-radio.net/2014/05/episode-204-anil-madhavapeddy-on-the-mirage-cloud-operating-system-and-the-ocaml-language/
• http://openmirage.org (official site)