I have 2 machines at home, one is a pine64 running a Debian linux and a desktop PC with windows 8.

I successfully installed openVPN server to the pine64 so I have a working setup, the openVPN service is accessible from the local network through the local IP address of the server, I tested the connection with my desktop PC.

The VPN is also working from the outside network through my router's WAN IP address, consequently the port 1994 is forwarded correctly to the openVPN host.

I also tested the connection from the outside network access with my cellphone (mobile network) and the openVPN connect client, everything went fine.

I would like to simulate/test the VPN access as it was an outside network from my desktop PC.
For example I want to check whether I could access my other other hosts in the network through SSH if I will be far away from my home network.

What I don't quite understand is why I cannot access my VPN server from the local network through the router's public WAN IP.

The 2 machines have static IPs on the same network:

desktop PC: 192.168.1.11

pine 64 (openVPN server): 192.168.1.20

let the router's public WAN IP be (for the sake of the example): 5.39.182.24

So I'm trying to access the openVPN server with the IP 5.39.182.24:1194, but unfortunately I am not able to.
There's no firewall setup on the PC or any other application I aware of that could disallow the connection.
Trying the same approach with my cellphone from the local network fails too, so it's proven that it is not strictly an issue of the desktop machine.

Here's the log I get from the openVPN client application

Mon Sep 12 20:31:08 2016 OpenVPN 2.3.12 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Aug 23 2016
Mon Sep 12 20:31:08 2016 Windows version 6.2 (Windows 8 or greater) 64bit
Mon Sep 12 20:31:08 2016 library versions: OpenSSL 1.0.1t 3 May 2016, LZO 2.09
Mon Sep 12 20:31:13 2016 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Mon Sep 12 20:31:13 2016 Control Channel Authentication: tls-auth using INLINE static key file
Mon Sep 12 20:31:13 2016 Attempting to establish TCP connection with [AF_INET]5.39.182.24:1194 [nonblock]
Mon Sep 12 20:31:23 2016 TCP: connect to [AF_INET]5.39.182.24:1194 failed, will try again in 5 seconds: Connection timed out (WSAETIMEDOUT)
Mon Sep 12 20:31:38 2016 TCP: connect to [AF_INET]5.39.182.24:1194 failed, will try again in 5 seconds: Connection timed out (WSAETIMEDOUT)

Server side settings

openVPN config

root@pine64:/etc# cat /etc/openvpn/server.conf
local 192.168.1.20 # SWAP THIS NUMBER WITH YOUR RASPBERRY PI IP ADDRESS
dev tun
#proto udp #Some people prefer to use tcp. Don't change it if you don't know.
proto tcp
port 1194
ca /etc/openvpn/easy-rsa/keys/ca.crt
cert /etc/openvpn/easy-rsa/keys/pine64.crt # SWAP WITH YOUR CRT NAME
key /etc/openvpn/easy-rsa/keys/pine64.key # SWAP WITH YOUR KEY NAME
dh /etc/openvpn/easy-rsa/keys/dh2048.pem # If you changed to 2048, change that here!
server 10.8.0.0 255.255.255.0
# server and remote endpoints
ifconfig 10.8.0.1 10.8.0.2
# Add route to Client routing table for the OpenVPN Server
push "route 10.8.0.1 255.255.255.255"
# Add route to Client routing table for the OpenVPN Subnet
push "route 10.8.0.0 255.255.255.0"
# your local subnet
push "route 192.168.1.20 255.255.255.0" # SWAP THE IP NUMBER WITH YOUR RASPBERRY PI IP ADDRESS
# Set primary domain name server address to the SOHO Router
# If your router does not do DNS, you can use Google DNS 8.8.8.8
#push "dhcp-option DNS 192.168.2.1" # This should already match your router address and not need to be changed.
push "dhcp-option DNS 8.8.8.8" # This should already match your router address and not need to be changed.
# Override the Client default gateway by using 0.0.0.0/1 and
# 128.0.0.0/1 rather than 0.0.0.0/0. This has the benefit of
# overriding but not wiping out the original default gateway.
push "redirect-gateway def1"
client-to-client
duplicate-cn
keepalive 10 120
tls-auth /etc/openvpn/easy-rsa/keys/ta.key 0
cipher AES-128-CBC
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn-status.log 20
log /var/log/openvpn.log
verb 1

iptables

(exported the rules to a file with iptables-save)

root@pine64:/etc# cat /etc/iptables-firewall-rules.backup
# Generated by iptables-save v1.4.21 on Sun Sep 11 21:19:15 2016
*filter
:INPUT ACCEPT [16429:2363941]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [17426:8592638]
-A INPUT -i eth0 -p udp -m state --state NEW -m udp --dport 1194 -j ACCEPT
-A INPUT -i tun+ -j ACCEPT
-A FORWARD -i tun+ -j ACCEPT
-A FORWARD -i tun+ -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o tun+ -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o tun+ -j ACCEPT
COMMIT
# Completed on Sun Sep 11 21:19:15 2016
# Generated by iptables-save v1.4.21 on Sun Sep 11 21:19:15 2016
*nat
:PREROUTING ACCEPT [1172:103090]
:INPUT ACCEPT [157:31732]
:OUTPUT ACCEPT [205:14166]
:POSTROUTING ACCEPT [205:14166]
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT --to-source 192.168.1.20
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT --to-source 192.168.1.20
COMMIT
# Completed on Sun Sep 11 21:19:15 2016

Output of the route command

root@pine64:/etc# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default speedport.ip 0.0.0.0 UG 0 0 0 eth0
10.8.0.0 10.8.0.2 255.255.255.0 UG 0 0 0 tun0
10.8.0.2 * 255.255.255.255 UH 0 0 0 tun0
link-local * 255.255.0.0 U 1000 0 0 eth0
192.168.1.0 * 255.255.255.0 U 0 0 0 eth0

I hope someone could shed some light on this issue, I appreciate the help.

You can not connect your VPN from the lan using the external IP because if you think about it for a second the purpose of the VPN is to connect two (or more) machines that are not in the same network but in your case they already are on the same network.