Antsle Forum

Welcome to our Antsle community! This forum is to connect all Antsle users to post experiences, make user-generated content available for the entire community and more. 

Please note: This forum is about discussing one specific issue at a time. No generalizations. No judgments. Please check the Forum Rules before posting. If you have specific questions about your Antsle and expect a response from our team directly, please continue to use the appropriate channels (email: [email protected]) so every inquiry is tracked. 

Please or Register to create posts and topics.

[SOLVED] Bizarre Antsle Network Issues - Can't update, Can't Port Forward

akak01000101@akak01000101
67 Posts
#1 · October 5, 2019, 2:47 pm

Running the Antsle (edgeLinux 0.5.0) on my home network. I know it cannot be related, but as soon as I patched and reboot a Ubuntu LXC I started seeing weird network issues with my Antsle:

  • I lost http connectivity from my wifi enabled laptop to AntMan (0.11.97947208069579961a) that I was working from.
  • I cannnot connect to my Antsle via a ssh connection directly from my Wifi laptop.
  • I cannot connect to any Antlets via ssh (-p 22xxx)
  • I can ssh to Antlets over 192.168.1.x on my local NAT'd network.
  • From an Antlet, I can connect to the Antsle over 10.1.1.x
  • If I use a computer with a wired network connection, I can connect to AntMan and ssh directly with no issues.
  • While connected via ssh to the Antsle with a shell, I cannot ping out to the Internet, only devices within the local network.
  • All my Internet based websites that my router port forwards through the Antsle do not connect, but they are available from the NAT'd 192.168.x.x network.
  • There have been no changes to my router in over a month
  • All Antlets seem to be performing as they all should. They are having no network issues.

From the Antsle, if I login with a shell and ping something on the internet, it resolves to my router. I wish nslookup or dig was installed. I can connect to http with the nc command to my web servers on the Local Network, but not using DNS. Other systems point resolv.conf to my router and resolve DNS just fine.

# ping yahoo.com
PING yahoo.com (192.168.1.1) 56(84) bytes of data.

 

# cat /etc/resov.conf

nameserver 192.168.1.1

 

Here's the routing table.

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default localhost 0.0.0.0 UG 8 0 0 br0
localhost 0.0.0.0 255.255.255.0 U 0 0 0 virbr1
localhost 0.0.0.0 255.255.255.0 U 0 0 0 virbr2
localhost 0.0.0.0 255.255.255.0 U 8 0 0 br0
localhost 0.0.0.0 255.255.255.0 U 0 0 0 virbr0

I also thought if I try to upgrade antman, it may clean something up, and I get the following error. I suspect cause it can't get out to the internet. An upgrade of edgeLinux yields similar results.

# upgrade-antman
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib64/python2.7/json/__init__.py", line 291, in load
**kw)
File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
--2019-10-05 14:29:08-- http://central.antsle.com/antman-releases//amanupd-
Resolving central.antsle.com... 192.168.1.1
Connecting to central.antsle.com|192.168.1.1|:80... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2019-10-05 14:29:08 ERROR 503: Service Unavailable.

chmod: cannot access 'amanupd-': No such file or directory
/usr/local/bin/upgrade-antman: line 11: ./amanupd-: No such file or directory
rm: cannot remove 'amanupd-': No such file or directory
Error while upgrading antman
* WARNING: antman has already been started

 

Any help on a direction to check is GREATLY appreciated!

Click for thumbs down.0Click for thumbs up.0
Last edited on October 10, 2019, 8:55 am by akak01000101
mardo.delcid@mardo
75 Posts
Antsle Staff
#2 · October 8, 2019, 11:23 am

Hi @akak01000101,

Looks like something is blocking central.antsle.com, please try:

curl http://central.antsle.com/api/antman-info

The response should be a JSON string that looks like this:

{"version":"2.1.0c","edgeLinux-version":"2.1.0c","anthilld-version":"1.2.7","early-access-promo-due":1535525940000}

Do you have a firewall blocking central.antsle.com?

Click for thumbs down.0Click for thumbs up.0
akak01000101@akak01000101
67 PostsTopic Author
#3 · October 8, 2019, 12:57 pm

Multiple systems can connect through the firewall no problem to central.antsle.com and return:

{"version":"2.1.0c","edgeLinux-version":"2.1.0c","anthilld-version":"1.2.7","early-access-promo-due":1535525940000}

I'm thinking a routing issue somewhere in the Antsle:

root@myantsle:/tmp # wget http://central.antsle.com/api/antman-info
--2019-10-08 12:51:32-- http://central.antsle.com/api/antman-info
Resolving central.antsle.com... 192.168.1.1
Connecting to central.antsle.com|192.168.1.1|:80... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

--2019-10-08 12:51:34-- (try: 2) http://central.antsle.com/api/antman-info
Connecting to central.antsle.com|192.168.1.1|:80... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2019-10-08 12:51:34 ERROR 503: Service Unavailable.

root@myantsle:/tmp # curl -vvv http://central.antsle.com/api/antman-info
* Trying 192.168.1.1...
* TCP_NODELAY set
* Connected to central.antsle.com (192.168.1.1) port 80 (#0)
> GET /api/antman-info HTTP/1.1
> Host: central.antsle.com
> User-Agent: curl/7.54.1
> Accept: */*
>
* Empty reply from server
* Connection #0 to host central.antsle.com left intact
curl: (52) Empty reply from server

Click for thumbs down.0Click for thumbs up.0
mardo.delcid@mardo
75 Posts
Antsle Staff
#4 · October 8, 2019, 3:35 pm

Hi @akak01000101,

Looking at the curl command output, I think the `upgrade-antman` command will work now. If so, could you please check if the issues you listed above still happen?

Click for thumbs down.0Click for thumbs up.0
akak01000101@akak01000101
67 PostsTopic Author
#5 · October 10, 2019, 8:54 am

Thanks Mardo! Looks like your recommendation worked when I comment out the following lines in /etc/conf.d/net

#bridge_forward_delay_br0=0
#bridge_hello_time_br0=200
#bridge_stp_state_br0=1
Then run:
# service net.br0 restart
Click for thumbs down.0Click for thumbs up.0
akak01000101@akak01000101
67 PostsTopic Author
#6 · October 26, 2019, 5:24 pm

Restarting the net.br0 was a temporary fix. To truly fix the issue I needed to do a complete reset of edgeLinux. Instructions below were provided by Antsle support:

CLICK ON THIS WINDOW TO EXPAND THE INSTRUCTIONS

The following instructions how to reset the edgeLinux OS to version 0.12.1. antlets will remain and the root password will be set to 'antsle'.

**This will remove all docker images and containers.

Using 2 USB sticks is the easiest method. The first will be a bootable USB stick, we recommend using SystemRescueCD:
http://www.system-rescue-cd.org/Installing-SystemRescueCd-on-a-USB-stick/
I use a utility called Rufus to create the bootable usb with the sysrescue .iso (https://rufus.akeo.ie)

The second should be at least 32G capacity and formatted with NTFS or exfat file system as one of the uncompressed images is ~16G.

Connect a monitor and keyboard or connect to the terminal via IPMI: https://docs.antsle.com/setup/#4-ipmi

Remove all other USB storage devices.
Boot the antsle from the bootable USB.
Run the command

lsblk

Your bootable USB should be the last device (sdc)
Plug in the 2nd USB and run 'lsblk' again. This USB should now be the last device (sdd). This USB will probably only have 1 partition but may have two. We will need the one using all the storage space (see the 'size' column). I'll use 'sdd1' in the example.

Lets mount the 2nd USB

mkdir /usb2
mount /dev/sdd1 /usb2

Change into the /usb2 directory

cd /usb2

Download the reset script, make it executable, and run the script:

wget https://static-files.antsle.com:8443/edgeLinux/0.12.1/el-reset
chmod +x el-reset
./el-reset

When the script finishes, power off the antsle, remove the USB sticks and reboot.

** The new root password will be 'antsle'.

Login at the prompt and run the command

el-reset-phase2

 

Click for thumbs down.0Click for thumbs up.0
Last edited on November 9, 2019, 9:25 am by akak01000101