Quote from daniel.luck on January 1, 2023, 7:44 pm

Hi @kryten

Sorry for the late reply.  I believe that I had worked with someone to get Power Supply ordered.

Here's the link for the Power Supply for our antsle One for anyone interested:
https://shop.antsle.com/product/power-supply-antsle-one/

Thanks,
antsle Support

Quote from kryten on January 2, 2023, 5:40 pm

Hey

 

So i got a new PS and the machine started up again and ran for a couple of days... Now i have this issue where i cant start any antlets... It says they successfully started but they do not... I have rebooted the server but no change... I am not able to restart the console either,  so cant restart antman

Any suggestions?

Quote from kryten on January 2, 2023, 6:11 pm

Also

 

I noted that under system services that docker failed to start and wont restart

 

Quote from daniel.luck on January 3, 2023, 2:37 pm

Hi @kryten

Thanks for the update.  Sorry to hear that you cannot start any of your antlets.

Using the command-line, can you use these commands and let us know output?  Sometimes, we can see more details in SSH when starting up antlets.

For KVM's:
virsh start antlet_name

For LXC's:
vc start antlet_name

where antlet_name is the name of our antlet

Here are some instructions below on getting the docker service working again.

This procedure will get docker service working again but WILL WIPE OUT existing Docker Containers:

Stop and wipe all current docker files:
service docker stop
rm -rf /var/lib/docker/*

Try to mount:
zfs mount antlets/_docker

If the command above fails saying antlets/_docker doesn't exist, run:
zfs create antlets/_docker -o mountpoint=/var/lib/docker

check that /etc/docker/daemon.json contains { "storage-driver": "zfs" }
cat /etc/docker/daemon.json

Start docker:
service docker start

Check docker Storage is using zfs
docker info | grep zfs

You can get the default docker images by running the upgrade-antman command:
upgrade-antman

Make sure the docker service starts after a reboot
eL 2.0.0: systemctl enable docker
eL 0.x.x: rc-update add docker default

Thank you,
antsle Support

Quote from kryten on January 3, 2023, 3:18 pm

Hi Daniel

Thanks…

Following your instructions

Command: virsh start antlet_name

Results: It says it has started but when I look at antman it displays “Resume” but I cant resume it from there.

I then tried

Command: service docker stop
Command: rm -rf /var/lib/docker/*
Command: zfs mount antlets/_docker

It says that antlets already mounted. So I could not continue

See image 1

I did check the { "storage-driver": "zfs" } it is there

 

Then I tried

Command: service docker start

Fails – see image 2

In Antman when I look at services Docker is in failed condition and will not restart

See image 3

Also when i try to start console in Antman it fails to start as well... It says "Docker Service Stopped"

Quote from kryten on January 4, 2023, 4:37 pm

Hi Daniel

I worked on this today.. I found that there were two items with issues docker. Service and docker.socket . both show up as failed.... as i dug deeper i did a "journalctl -u docker. Service -b" and it said that docker could not resize a file and mentioned issue with space... I did not capture the actual message(sorry)... So even though nothing had changed in the last month or so other than the PS failure, it looked like i had only about 14gb of space. So to get some more space, i deleted an antlet, which gave me another 30'ish gb and rebooted... everything turned to normal, I could restart my antlets (i only have 4) and it worked... However, after 2 hours of running I noticed that my 30gb of space turned into 160gb of space, which is much more like what I used to see... Maybe the PS failure caused an issue to use space by crashing it.

Anyway I wanted to let other "newbies" know incase they have similar issues... Let me know if you want more info here. But i think it is solved

p

Quote from daniel.luck on January 4, 2023, 4:44 pm

Hi @kryten

Thanks for the update.  I may have had some missing steps.

Before trying to remove docker, try either of these commands to stop antMan:

For edgeLinux 2.0 or higher:  systemctl stop antman
For edgeLinux 0.12.0 or lower:  service antman stop

Can you let us know what version of antsleOS/edgeLinux and antMan you are running?

antsleOS-version
antman-version

If you are using edgeLinux 2.0 or higher, you can use this command to upgrade/re-install latest antMan:

upgrade-antman

If you are using edgeLinux 0.12.0 or lower, you can use procedure below to upgrade/re-install latest antMan:

If so, you will need to download the antMan bundle from your anthill.antsle.com account in the Downloads page - antMan 3.4.3 for x86_64
Copy the file to your antsle using scp or an sftp client like Filezilla (https://docs.antsle.com/system/secure-ftp-sftp-access)

Or download it directly to your antsle with 'wget'. Right click the Download link and copy the link address. Then SSH to the antsles command line and run the 'wget' command and paste the link e.g:
wget https://s3.wasabisys.com/wasabi--cloud-6a2aa848-8fea-4d97-b476-2e5ef9db7c18/antman-updates/antman-3.4.3-x86_64.tar.gz

Your link may be different than this example so be sure and copy the link from antHill.

On the antsle's command line, extract the file
tar -xzf antman-X.X.X-x86_64.tar.gz

CD into the new directory
cd antman-x.x.x-x86_64

and run the installer
./install

Thanks,
antsle Support

 

Quote from kryten on January 4, 2023, 5:31 pm

Hi Daniel

I am running

edgeLinux 2.0
antman 3.4.3

and i ran upgrade-antman
with successful results..

What else did you need me to do?

Quote from daniel.luck on January 4, 2023, 6:04 pm

Hi @kryten

Thanks for the update.

Were you able to get docker working by following instructions above?

The previous instructions didn't work because I forgot you had to stop antMan before trying to re-install docker.

systemctl stop antman

Then follow the instructions in previous post.

Thanks,
antsle Support