[workspace-user] workspace deployment hangs - unexplained state change: Unstaged --> Corrupted

Fri Mar 23 21:43:07 CDT 2007

On Sat, 24 Mar 2007 00:17:03 +0100
Manuel Rodriguez Pascual <supermanue at gmail.com> wrote:

> Hi everyone,
> 
> Hello all,
> 
> I sent this mail to gt-user but was advised to send it here. So here  
> we go...
> 
> 
> I'm a newie using globus. I think I have everything corectly  
> configured, but jus't cannot make it work. I am using debian on the  
> server, and a debian Virtual Machine. The VM was (more or less)  
> created as explained in http://www.howtoforge.com/ 
> perfect_setup_xen3_debian . I think it is a standard setup, with  
> nothing strange. It works correctly when being run with Xen, except  
> doesn't have internet configurated.
> 
> Globus is correctly instalated and being used by many people, so I  
> supose the problem is not there.
> 
> I was wondering if this issue is something you've seen before, and if  
> so whether you could recommend tests or checks to find the source of  
> the problem. Bellow I have posted everything I think may help to find  
> my problem.
> 
> Thanks for your attention,
> 
> Manuel Rodriguez Pascual
> 
> 
> I enter the following command:
> 
> 
> > workspace --file workspace.epr --metadata /usr/local/globus/share/ 
> > workspace_client/sample-workspace.xml -s https://localhost:8443/ 
> > wsrf/services/WorkspaceFactoryService --deploy-duration 30 --deploy- 
> > mem 256 --deploy-state Running
> >
> 
> 
> And I get the following output:
> 
> 
> > Using endpoint:
> > Address: https://localhost:8443/wsrf/services/WorkspaceFactoryService
> > Reference property[0]:
> > <ns1:WorkspaceKey xmlns:ns1="http://www.globus.org/2006/08/ 
> > workspace" xmlns:xsd="http://www.w3.org/2001/XMLSchema"  
> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
> > xsi:type="xsd:string">default</ns1:WorkspaceKey>
> >
> > Reading in metadata ... ok.
> > Creating deployment request from arguments... ok.
> >
> > *** Deployment request:
> >
> >     - Node number: 1
> >     - minDuration: 1800 seconds
> >     - State: Running
> >     - Default shutdown mechanism: Normal
> >     - individualPhysicalMemory:
> >         - exact: 256.0
> >
> > *** Classloading for a while, then creating workspace "http:// 
> > example1/localhost/image"... ok.
> >
> > Resource key: 164
> > Instantiation time: Fri Mar 23 19:56:40 CET 2007
> > Duration: 1800 seconds (roughly 30 minutes)
> > Shutdown time: Fri Mar 23 20:26:40 CET 2007
> > Resource Termination time: Fri Mar 23 20:56:40 CET 2007
> >
> > Wrote EPR to 'workspace.epr'
> > Subscribed to termination notification.
> >
> > Subscribed to deployment changes. Waiting.
> >
> > *** Deployment:
> >
> >     - State changed: Unstaged --> Corrupted
> >
> >
> > *** Network configuration:
> >
> >     - NIC #1
> >     - ------------
> >     -             Name: eth0
> >     -              MAC: ANY
> >     -     Network mode: Bridged
> >     - IP configuration: Allocate
> >     -       IP address: 192.168.0.12
> >     -       IP gateway: null
> >
> >
> 
> 
> I can terminate it without any problem, running
> 
> workspace -e workspace.epr --destroy
> 
> ---
> The metadata file (sample-workspace.xml) is the default file. I just  
> modified the VM I want to run, with the following sentence:
> 
> <def:uri>file:///local/home/manuel/base.img</def:uri>
> 
> 
> ---
> At last, this is the output information I recibe in globus/var/ 
> container.log.
> 
> 
> > 2007-03-23 21:36:57,406 INFO  factory.WorkspaceFactoryService  
> > [ServiceThread-1059,create:62] [WORKSPACE-EVENT]: WS-CREATE invoked  
> > by /O=UCM/OU=ASDS-DACYA/CN=Manuel Rodriguez
> > 2007-03-23 21:36:57,429 INFO  factory.AssociationUtil  
> > [ServiceThread-1059,getNextEntry:96] [WORKSPACE-EVENT]: 'public'  
> > association entry leased, ip=192.168.0.12
> > 2007-03-23 21:36:57,441 INFO  factory.ResourcepoolUtil  
> > [ServiceThread-1059,getResourcepoolEntry:92] [WORKSPACE-EVENT]:  
> > 'pool1' resource pool entry 'ursa02': 256 MB reserved, 127 MB left
> > 2007-03-23 21:36:57,451 INFO  service.WorkspaceHome  
> > [ServiceThread-1059,create:631] [WORKSPACE-EVENT][id-165]:
> >
> > WORKSPACE RESOURCE CREATED:
> >     - Name: 'http://example1/localhost/image'
> >     - Key: '165'
> >     - Start time:                Mar 23, 2007 9:36:57 PM
> >     - Shutdown time:             Mar 23, 2007 10:06:57 PM
> >     - Resource termination time: Mar 23, 2007 10:36:57 PM
> >     - Creator DN: /O=UCM/OU=ASDS-DACYA/CN=Manuel Rodriguez
> >
> > 2007-03-23 21:36:57,481 INFO  impls.WorkspaceResourceImpl  
> > [Timer-0,setOpsEnabled:370] [WORKSPACE-EVENT][id-165]:  WS- 
> > operations enabled
> > 2007-03-23 21:36:57,529 INFO  workspace.WorkspaceUtil  
> > [Thread-18_WorkspTaskThrd,runCommand:155] [WORKSPACE-EVENT]: /usr/ 
> > bin/ssh xenadmin at ursa02 /opt/workspace/bin/workspace-control -- 
> > create --name workspace-165 --memory 256 --networking  
> > 'eth0;public;ANY;Bridged;Static;192.168.0.12;null; 
> > 192.168.0.255;255.255.255.0;192.168.0.1;null;null;null;null;null' -- 
> > image file:///local/home/manuel/base.img --imagemount sda1
> > 2007-03-23 21:36:58,788 INFO  workspace.WorkspaceUtil  
> > [Thread-18_WorkspTaskThrd,runCommand:176] [WORKSPACE-EVENT]: Return  
> > code is 2
> > 2007-03-23 21:36:58,789 ERROR workspace.WorkspaceUtil  
> > [Thread-18_WorkspTaskThrd,runCommand:232] [WORKSPACE-EVENT]: system  
> > command FAILURE
> > STDERR:
> >
> > 2007-03-23 21:39:20,797 - ERROR - workspace (1072) - Problem  
> > validating configuration:
> > Traceback (most recent call last):
> >   File "worksp.py", line 455, in main
> > IOError: [Errno 28] No space left on device

Hi there, 

This is the issue here, "No space left on device", there is no space left to
write on the node's disk.  You need to clear up some space by deleting
unecessary files.  

The workspace-control program writes to a small persistence file to track some
things, it looks like this is where it is failing. 

Tim 

> > 2007-03-23 21:36:58,789 INFO  xen.XenTask  
> > [Thread-18_WorkspTaskThrd,execute:126] [WORKSPACE-EVENT][id-165]:  
> > Start failed
> > 2007-03-23 21:36:58,789 ERROR impls.StatefulResourceImpl  
> > [Thread-18_WorkspTaskThrd,notify:123] Problem moving [id-165] to  
> > state 'Started'
> > org.globus.workspace.WorkspaceException: Problem with parameters to  
> > workspace driver, incorrectly supplied or validation of request failed
> >         at org.globus.workspace.xen.XenUtil.throwErr(XenUtil.java:485)
> >         at org.globus.workspace.xen.XenUtil.translateReturnException 
> > (XenUtil.java:449)
> >         at org.globus.workspace.xen.XenTask._execute(XenTask.java:203)
> >         at org.globus.workspace.xen.XenTask.execute(XenTask.java:105)
> >         at  
> > org.globus.workspace.service.impls.async.WorkspaceThread.run 
> > (WorkspaceThread.java:56)
> > 2007-03-23 21:36:58,791 WARN  impls.StateTransition  
> > [Thread-18_WorkspTaskThrd,corrupted:232] Workspace was corrupted  
> > (when moving to state Started): can not change state anymore unless  
> > workspace is going to be destroyed
> >
> 
> 