SmartQuant Discussion

Automated Quantitative Strategy Development, SmartQuant Product Discussion and Technical Support Forums
It is currently Tue Nov 21, 2017 12:12 am

All times are UTC + 3 hours




Post new topic Reply to topic  [ 15 posts ] 
Author Message
PostPosted: Thu Jul 27, 2017 7:04 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
I'm trying to connect my desktop OpenQuant v3.7.1 to QuantBase 2.2.1, which runs on AWS. But I get this error on all my attempts:

Code:
System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond nnn.nnn.nnn.nnn:5333  [b](<= 5333 is the port number configured on both OpenQuant and QuantBase. nnn is the IP of my AWS machine)[/b]

Server stack trace:
   at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
   at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket(EndPoint ipEndPoint)
   at System.Runtime.Remoting.Channels.SocketCache.GetSocket(String machinePortAndSid, Boolean openNew)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at SmartQuant.QuantBaseLib.IQuantBaseConnection.Close()
   at SmartQuant.QB.QuantBase.Disconnect()


I believe that I have all the firewall ports open correctly since I can telnet to port 5333 on the QuantBase machine and connect. But it terminates the connection as soon as I type something. (It says ".NET<non-ascii chars>Server encountered an internal error. To get more info turn on customErrors in the server's config file.")

I believe I have the OpenQuant connection syntax correct because I can connect to my desktop QuantBase easily. To connect to the remote machine, I replace "localhost" with the IP or DNS name of the AWS machine running QuantBase. I have tried connecting with and without a username/password without success.

I run Windows10x64 on my desktop, and Windows Server 2008 R2 on AWS.

Does anyone have any ideas on what the problem might be? Thank you.


Attachments:
ab.png
ab.png [ 18.58 KiB | Viewed 802 times ]
aa.jpg
aa.jpg [ 24.2 KiB | Viewed 802 times ]
Top
 Profile  
 
PostPosted: Mon Jul 31, 2017 10:30 am 
Offline

Joined: Wed May 05, 2010 9:49 pm
Posts: 383
Hello kkkwj,
I have not access to AWS instances, so I checked connection using local network.
It works properly.

Perhaps problem in old version of software, you can try update them and check again.
Otherwise you should check network settings.
Additionally you can try to change port number.


Attachments:
QuantBase connection test.png
QuantBase connection test.png [ 322.26 KiB | Viewed 782 times ]
Top
 Profile  
 
PostPosted: Mon Jul 31, 2017 10:46 am 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Hi, Thank you for your help.

I did try to update my software, but the link in the update box did not link me to a new version of the software.

I also can get OpenQuant to connect to QuantBase on my local machine with no problem.
I have tried changing ports, checking all the server firewall ports, the AWS firewall ports, etc.

It should be so simple, and yet it doesn't work.

Is the same syntax used for local and remote connections?

tcp://hostname:port/QB


Top
 Profile  
 
PostPosted: Mon Jul 31, 2017 10:50 am 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
PS. I should ask, is there a link you can give me for newer (or the latest) versions of OpenQuant, QBase, router, trader, etc? (Not the new Openquant 2014--my license won't work for those new programs, I think.) Anton knows the license I have, if you have questions. Thank you


Top
 Profile  
 
PostPosted: Mon Jul 31, 2017 10:56 am 
Offline

Joined: Wed May 05, 2010 9:49 pm
Posts: 383
I sent links to PM.
I nothing changed in settings of QuantBase.
In OQ3 I just set ip of the machine with QuantBase.


Top
 Profile  
 
PostPosted: Mon Jul 31, 2017 4:32 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
To be clear, it looks to me like

(1) OQ on my desktop can connect to QB on my AWS server.
This is because the green light shows on QB in the provider list when I connect.

(2) But when I try to use QuantBase Explorer to get a list of time series (by clicking the refresh icon), I get the error message. I use 'tcp://aws-host-ip/QB' as the connection string in the QB Explorer connect dialog and leave the username and password blank. (I have tried filling them in too, but with no success.)

(3) If I try to use the import dialog to request a list of instruments, I also get the error message.

(4) All of these operations work fine on my desktop OQ and QBase combination.
I turned off the Windows firewall on my AWS server for both public and private networks, and defined a QuantBase incoming rule that allows inbound connections from both private and public networks, and made sure the port numbers on my desktop OQ (5333) matched the AWS firewall inbound rule (5333) and the Windows server rule (5333). But still...no happiness.

Does that extra information help?

(5) Does the green light in the providers list really mean that OQ has connected successfully to QBase through all firewalls? If that is true, I'm wondering if a second data port must be opened in the firewalls to request and send data?

Thank you

UPDATE:

I have a tiny little client/server program pair that I use for testing TCP connectivity between machines.
I put the server on the AWS machine, and ran the client on my desktop.
They could talk to each other just fine through all the firewalls.
So I'm confident that I have the firewall rules set properly for QBase on the AWS server.

If I can trust the green light on OpenQuant, and the OQ Properties panel status saying "Connected=true", then it looks like a data request issue that only shows up between the client (OQ) and server (QBase) when a request is made...

Any ideas that you have for testing or fixing the issue would be appreciated.. Thank you


Top
 Profile  
 
PostPosted: Thu Aug 17, 2017 5:51 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
I was finally able to get OQ 3.17 (the latest version) running on my desktop (my license file was in the wrong place).

Now I am running QuantBase 2.2.1 32bit and OQ 3.17 64bit.

In OpenQuant I enter the public IP of QuantBase on the AWS server, right-click the icon to connect, and then I see:
- a green light on the QuantBase provider icon
- "IsConnected = True" in the Properties panel for the QBase provider
- "Status=Connected" in the Properties panel for the QBase provider

(1) Do those indicators really mean I'm connected to QBase?

(2) Because when I try to request a list of instruments or to see data from QB, the request dialog hangs for a long time and then fails, as described above.

My network test program has no problem going through the firewalls, as described above, so I think the firewall rules and ports (5333) are all configured properly. And OQ says that it is connected to QuantBase, and I'm running the latest versions.

Do you have any other ideas? The only thing I haven't tried recently (I think I tried it earlier) is to reboot QuantBase. But I'm not sure that would be helpful. OpenQuant on AWS (same machine) has no trouble connecting to QuantBase on that machine. The green light says the connection is OK, but the data requests fail.

I wonder what is different between the green light connection packets and the data requests? Different ports? UDP vs TCP? Something else?


Top
 Profile  
 
PostPosted: Thu Aug 17, 2017 8:42 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
I can also give you the C source code for my little TCP socket testing program if it would save you time.

In the old days when I ran a software company, we had a variety of little console test programs that would print out every step of a client-server connection--interaction process so that we could send it to a customer so they could see (and log) exactly where the problem was.

Maybe you have such a program that I could run?

Thank you
Kevin


Top
 Profile  
 
PostPosted: Fri Aug 18, 2017 12:43 pm 
Offline

Joined: Wed May 05, 2010 9:49 pm
Posts: 383
QuantBase version 2.2.1 is pretty old, latest version is 2.13,
try update it and check.


Top
 Profile  
 
PostPosted: Fri Aug 18, 2017 4:45 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Ok, I will try to upgrade later today and this weekend.
My desktop QBase v2.12.0 reports that there are no updates (no v2.13 to download).
I will use that version unless you can send me a link to v2.13...
Thanks


Top
 Profile  
 
PostPosted: Sat Aug 19, 2017 1:48 am 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Hi, I upgraded QuantBase on my AWS server to v 2.12.0 64 bit.
The license is in the root directory beside the executable.
QuantBase v2.12 loads properly and lets me see my data.

OpenQuant 3.17 on the AWS server connects to QBase just fine.
It uses localhost, port 5333, which is the default port for QBase.
I can request a list of instruments and get an instant response.
No problems.

On my desktop, OQ and QB can talk to each other easily, port default port 5333 again.

BUT desktop OQ 3.17 64bit and AWS QuantBase 2.12 64 bit do not work together.
OQ shows a green light on the icon, IsConnected=true, Status=Connected.
But OQ hangs and reports a socket error (as described above) when I request a list of instruments.

From my desktop I can 'telnet <aws ip address> 5333' and get a connection (to QB, I assume).
But when I type something I get an error and the connection is closed (because QB does not speak telnet protocol).
So, I think the firewall ports are open properly.

My little TCP client-server test program has no problem with firewalls when configured on port 5333.
See the attached images: I sent the text 'dir' to the AWS server on port 5333 and received a correct response.
So, I think the firewall ports are open properly.

Here is output from my TCP client test program on my desktop:
Attachment:
aa.png
aa.png [ 8.01 KiB | Viewed 688 times ]


Here is output from my TCP server test program on the AWS server:
Attachment:
ab.png
ab.png [ 2.61 KiB | Viewed 688 times ]


I don't know what else to try. Both programs communicate on their local hosts. All correct network ports are open. My TCP test program shows the TCP ports are working on the default port 5333. I'm running the latest programs, but desktop OpenQuant 3.17 will not get data from QuantBase 2.12 on AWS.

How else can we debug the connection? (I'm hoping you have a client test program that I can run on my desktop to print out logging information about the connection.)


Top
 Profile  
 
PostPosted: Mon Aug 21, 2017 5:28 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
I had another idea over the weekend. AWS mini-servers are free for a year or so, so you could test it using your own AWS account.


Top
 Profile  
 
PostPosted: Tue Aug 22, 2017 5:21 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
I think I have a good guess why OQ can't talk to QuantBase on AWS servers.
AWS uses internal IP addresses for its zillions of machines, with IP addresses like 10.185.123.456.
They do that so they can manage and change the IP addresses as required, I think.

But they publish a non-changing public IP for each machine and forward traffic to the internal IP address.
I use the public IP in OpenQuant.

But when OpenQuant eventually prints a provider error about the connection, it prints the AWS internal IP address in the error message, not the public address. OpenQuant should never be using that internal IP address; it should always send traffic to the public IP address.

My test TCP client/server programs do nothing fancy with IP addresses--they use standard netstream() socket APIs, so they avoid the problem of trying to send to internal IP addresses that cannot be reached from the public internet.

I wonder if OpenQuant is doing something more advanced or non-standard, such as maybe extracting the sender's IP (the internal IP) from the incoming packet from the server, instead of continuing to use the original public IP from the OpenQuant QuantBase connector properties panel.

Could you please check if that is the case?

Thank you
Kevin


Top
 Profile  
 
PostPosted: Fri Aug 25, 2017 2:40 pm 
Offline

Joined: Wed May 05, 2010 9:49 pm
Posts: 383
I tested QuantBase placed on AWS and unfortunately it will not work over NAT.
QuantBase use .NET remoting technology and this not works over NAT/internet.
Theoretically you can find a way to pass connection between AWS and your local PC but I have not the experience with it.


Top
 Profile  
 
PostPosted: Fri Aug 25, 2017 5:23 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Ok. Thank you for trying Qbase on AWS.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 

All times are UTC + 3 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group