Clarion Handy Tools
HOME ABOUT NEWS ORDER SUPPORT
April 2017

April 29, 2017

CHT Web Data Server Robustness Test #5

We're completing a fairly lengthy code revision and optimization to several CHT Server modules that is paying some interesting ruggedness dividends in addition to providing us with some valuable feedback about what CHT servers can do.

In this most recent #5 test we set up the following on our remote server hardware, which if you remember, is located about 1000km from our location.

• Server HNDSLFSV.APP (a page and file server).
• We started two files uploading to it 171MB and 1103MB.
• We started the browser and requested a video be played from the server. (171MB)
• We started a second browser and requested a video be played (171MB)

So at this stage of the test, by way of stressing the server, it was busy receiving two files which combined were 1.3 GIG, AND the server was simultaneously busy uploading to our browsers, two files with a combined size of over 240MB.

Then, with all that work going on we then asked for dozens of web site pages to be displayed in yet another browser. Noting as we did so, the speed with which these pages were displaying.

Page turnaround in all cases was in the range of .03 seconds to .08 seconds depending on how many images were in the page. In many cases embedded images, can triple or quadruple the size of a page. Note too that IMAGES and .CSS files embedded in a page are each requested separately from the page. So that a single page can make a dozen or more socket requests before it can be rendered.

Remember that the page turnaround calculation includes the full time from when a request socket is opened with a request from the browser or client until the request socket is closed when it finishes delivering the page, image or request object.

We're now in the process of compiling and testing all example servers in our toolkit to make sure they work as expected.

This test was with HNDSLFSV.APP a CHT Page and File Server - (Type #4)

chtservertype4.png

Subsequent test will involve two CHT Client Servers - (Type # 3 ) named HNDCLIENTSVLEAN1VIEW.APP and HNDCLIENTSVLEAN4VIEW.APP

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 29th, 2017

 


April 28, 2017

CHT Web Data Server Robustness Test #4

Some more interesting results...

First a review of the objective:

While we're interested in using CHT servers for all kinds of purposes, and file uploading and downloading is included in those interests, the following tests involving HUGE file uploads and downloads are really only being allowed in order to determine the ability of a CHT server to handle the stress of HEAVY data transfers on the response times encountered by CHT Data Client apps.

We're also, herewith, documenting which parts of the data request/reponse process take the most amount of time, and are most likely to impact the server's ability to respond quickly to multiple CHT Client apps performing various data transactions with the data server.

We modified the server CHT Client Multi-Vw (HNDCLIENTSVLEAN1VIEW.APP) to accept large files, when requested, and to send large files, when requested. These are not tasks that a CHT Client Data Server would/should ever be asked to do; BUT it is a way of testing what limits the server can be pushed to before it becomes non-responsive or sluggish with data transaction responses.

Here are some things we're seeing...

We used as a benchmark a HNDPEOPLE_LBX.APP remote browse that asks for 1000 records (see next image)

stresstest_001.png

Under no-stress conditions these are the stats we get back from the server:

stresstest_002.png

The time it took from the moment the request for this browse was received (a receive socket was opened) until the time the request was fully completed and acknowledged by the client shows as 82/100ths of a second (i.e. 0.82 seconds).

Below that, on the second image, you can see that browse data package contained 56375 bytes and took 000000.00 seconds to transmit. The data transmission time was so low that a Clarion numeric picture of @N09.2 returned a value of 000000.00. Too fast for any kind of visible two decimal point resolution.

So socket turnaround time (0.82 seconds), from when the socket opened, to when it closed, involved these actions or steps taken by the server:

• 1) Read the request header from the client and parse out the query and view name.
• 2) Send the query to the server's V_PEOPLEVIEW procedure as directed.
• 3) Scan the V_PEOPLEVIEW for records matching the query (ID > 0) (max 1000)
• 4) Accumulate the data in a return buffer (a BSTRING containing XML)
• 5) Write the accumulated record-set to an XML file, compress and encrypt - in memory.
• 6) Call the UNTHREADEDFILESENDERA function to transmit the file to the client.

All of that action took place in 82/100ths of a second of which the return transmission was perhaps no more than say, 1/100th of a second.

Wow, we thought! How fast does that data transmission from server to client actually work?

Our next test was to use CHTSNAPGET.EXE to request a HUGE file from this server to see how long it would take to get that file back. Here are the results:

stresstest_003.png

Because the file was over 1MB, the server, this time, called THREADEDFILESENDER so that the server could process the file in the background in order to free the listen socket to return to the task of listening for more data requests from its clients.

Smaller files like our 56375 byte browse record-set are returned in-line by CHT server class functions, UNTHREADEDFILESENDERA or UNTHREADEDFILESENDERB.

The turnaround time of these smallish, data-return packages is so fast that it's not worth the effort and memory overhead to START() a Clarion thread.

The BIG test file, on the other hand, was 565,367,774 Bytes (i.e. over 1/2 GIG) and it took 3.86 seconds to send back to my desktop working on the LAN.

That's 141MB per second upload speed from the server!

Certainly this return speed from the server depends on the speed of the connection between the file request client and the server. And working locally on the LAN, the return speed capability, is unbelievable!

Now forget file upload/download for the time being, and consider data packages coming for the CHT client app requesting browse data.

When data is being requested from the server, these requests are queued back to back by the Windows Socket System.

So the requests are sequential and the responses are sequentially returned FIFO. Request/Response #1 must be completed (or be in threaded transmission mode) before Request/Response #2 (from the same or perhaps a different client) can be started.

Since data return packages are miniscule compared to these huge test file transmissions, it turns out that the return data transmission part of the equation, (see the bullet points above) is the SMALLEST part of the client-server-data return calculation.

With CHT Client Server data packages coming back compressed (and perhaps encrypted) and a browse with 1000 records (from our HNDPEOPLE_LBX.APP example) constitutes only, a measly 56375 bytes, the server can be given significant request workload from hundreds (perhaps thousands) of data clients like HNDPEOPLE_LBX.APP before it's performance is degraded to any extent.

If it becomes degraded, it's not the data return transmission part of a request/response transaction that's likely to cause a bottleneck.

We also found, that if the server is busy receiving data (as represented by 3 or 4 large files being uploaded to it) there is very little difference between the turnaround times for data transactions with the CHT client application.

That's all for this test which was working across our LAN.

Next time we will repeat a similar test across a WAN and across the public INTERNET.

We have some early stress test results involving large files combined with CHT Data Client requests across the public internet, and again, the results were waaay faster than we had ever dared to expect.

More about that next time, with CHT Web Data Server Robustness Test #5

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 28th, 2017

 


April 27, 2017

CHT Web Data Server Robustness Test #3

This test is a variation of the Client Server Data Stress test described in the earlier #2 post.

While this is a Q & D Test (i.e. Quickl and Dirty) it presents the tremendous promise of durability and ruggedness of our CHT Client Data Server technology.

Here's this test in brief.

We placed several copies of the same 171MB video in the "web" directory of CHT CLIENT SERVER demo HNDCLIENTSVLEAN1VIEW.EXE.

Then we started three tabs in our browser and asked on each tab, for the browser to start one of the videos mentioned above. When that task was completed we had three, 171MB videos, playing simultanteously on three separate tabs of our browser.

We then started HNDPEOPLE_LBX.APP and pointed it at our remote copy of HNDCLIENTSVLEAN1VIEW.EXE, and logged into that server with CHT's HNDPEOPLE_LBX.APP.

Once Logged in, we asked one of the "FillFromCHTServer" procedures to send us 200 records for the browse. Here's the server log portion for that data delivery:

HNDBrowserServer.SocketClose(1072) OK. Sockets Open: (1)
Turnaround time: (36/100 seconds)
Data sent: (13103 bytes)

The server delivered 200 browse records and 13103 bytes of data in 1/3 of a second. Even while 3 videos were playing (equivalent to downloading).

Next we upped the data request on my HNDPEOPLE_LBX.APP to 1000 records. Here's the server log portion of that data delivery:

HNDBrowserServer.SocketClose(1044) OK. Sockets Open: (1)
Turnaround time: (106/100 seconds)
Data sent: (55950 bytes)

The server delivered 1000 browse records and 55,950 bytes of data in 1 second. Even while 3 video were playing (i.e. uploading to our browser).

Remember as you look at these numbers that the turnaround time (1/3 second, and 1 second respectively) for these two remote browse queries INCLUDES the time it took to:

• 1) Scan the backend .TPS file (a multi-table-view) to find 1000 matching records.
• 2) Package the backendview records into an XML response file.
• 3) Upload the xml response file package up to HNDPEOPLE_LBX.APP

All this, we remind you, while the SAME server was busy uploading 3 videos of 171MB each.

This test took place, across the ordinary internet, at approximately 6:30PM EDT with a distance of approximately 1000km between my HNDPEOPLE_LBX.APP and the CHT Server application HNDCLIENTSVLEAN1VIEW.APP.

These response and delivery numbers across the internet are essentially as good as the response times you'd get if these files were local to your machine and the HNDPEOPLE_LBX.APP was running in local mode.

Amazing, those CHT Client Data Servers!!!

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 27th, 2017

 


April 26, 2017

CHT Web Data Server Robustness Test #2

We ran the second in an ongoing series of CHT Server Stress tests today and again, the results were very positive.

Today's test involved simultaneously sending a 171MB file to the server (uploading to the server with HNDSLFCMDXSA.EXE) and downloading a different 171MB file from the server using CHTSNAPGET.EXE. (See next image)

stresstest2_001.png

While that was going on, we used our desktop browser to view dozens of web pages loaded with quite large picture files.

The expectation (before testing) was that because the server was busy simultaneously uploading and downloading two 171MB files, that page requests would be slowed to a crawl because of the ongoing server data load.

We were amazed and surprised that the web pages being served to our browser were displaying as fast as ever. No apparent delay at all.

When we looked at the server log, we found these kinds of numbers:
EXITING THREADEDFILESENDER. SendSocket = (968) Upload Elapsed Time = ( 99 Seconds ) File =mars2.mp4
EXITING UNTHREADEDFILESENDERA. SendSocket = (1088) Upload Elapsed Time = ( 0 Seconds ) File = support.htm
EXITING UNTHREADEDFILESENDERA. SendSocket = (1088) Upload Elapsed Time = ( 0 Seconds ) File = purchase.htm
EXITING UNTHREADEDFILESENDERA. SendSocket = (1088) Upload Elapsed Time = ( 0 Seconds ) File = about.htm
ENTERING THREADEDFILESENDER. Socket = (968) File =mars2.mp4

This list is in reverse chronological order and does not include all of the browser activity, just a snippet of the files requested.

You can see from when the mars2.mp4 was requested (ENTERING THREADEDFILESENDER) until it was fully sent (EXITING THREADEDFILESENDER), took about 99 seconds (i.e. 171 MB file).

The page requests between these two points, support.htm, purchase.htm, about.htm including the pictures on these pages and perhaps 50 more pages, show an Elapsed delivery time of 0 seconds.

For this test we set time resolution output to minimum 1 second.

On closer examination the delivery times of these pages was always consierably under one second. (eg: 7/100's, 12/100's etc.)

Conclusion...

This bodes well for our next series of tests which will be to determine how much a CHT Client Data Server can be asked to send, by way of data, without feeling the stress of answering queries from multiple remote or local data clients.

The continuous stress of two 171MB files being uploaded/downloaded simultaneously is approximately 10,000 times as much work as is created by a single data request for a say, a 500 record browse. (estimated at 25,000-40,000 bytes).

So, in theory, 10,000 clients could be asking near-simultaneously to browse 500 records, and still there would be no appreciable delay in delivering the data packages (also delivered as XML files) back to those 10,000 client apps.

We hope you follow where we're going with this.

In an upcoming series of tests we'll be asking a CHT Client Data Server to do what this page server was asked to do (as described above) while data requests are sent to it from a CHT Client Server app.

That's all for now. More tests to come.

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 26th, 2017

 


April 25, 2017

CHT Web Data Server Robustness Test #1

We've been busy here testing (and improving where necessary) the robustness of our HTTP servers.

We wanted to know and see for ourselves, how a CHT server responded under heavy stress, by giving it multiple tasks to perform while asking it to carry on with normal duties such as serving pages, (which is really returning files) and returning Client Server data requests.

A test performed the other day, surprised even us, since we did NOT expect quite the amazing amount of responsiveness and resiliance that we were presented with.

To initially stress the server we began uploading three HUGE files to it: 171MB, 565MB and 1103MB, not separately one after the other, but near-simultaneously, using the HNDSLFCMDXSA.EXE uploader.

Here's how that looked on our desktop.

upload3videos.png

Our office connection is no barn-burner so the files took 2 minutes, 9 minutes and 17 minutes to upload, respectively. During the initial two minute interval while all three files were being received by the server, our rough calculations tell us it was proccessing input to itself, of approximately 200MB per minute.

For the next 5 minutes, while two files were uploading to it, the server was processing input to itself of approximately 126 MB per minute.

Then for the last 10 minutes, while the last 1.1 GIG file was uploading to it, the server was processing input to itself of approximately 64MB per minute.

During the interval that the server was being kept occupied receiving these VERY large amounts of data input, we accessed the server with a browser and requested the web pages of our website.

Amazingly, we didn't see any significant decrease in the speed that the server returned these web pages, including images, and control files embedded in those pages.

The average web-page turnaround time (see next image) was approximately 1/10th of a second per page, even while the server was under 200MB per minute download stress.

pagerequesttiming.png

This is only the first of a few tests we've designed and will document here on our What's New pages for April 2017.

We're next going to perform the same test with a CHT Client Data Server that is going to be asked to display browses and forms to a Clarion Client App. While simultaneously the server is accepting these same 3 large files.

Next, these two tests will be repeated, but in reverse, by asking these Servers to SEND us those 3 large files back, while we simultaneously ask for pages from a browser and/or we display Client Server browses being provided by the server app.

What we'll be documenting is the turnaround time of some busy web pages and the turn-around time of, say, a CHT Client server browse from ListBoxBrowseExtender of say 500 records and 1000 records.

Will keep you posted.

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 25th, 2017

 


April 1, 2017

Build Release 21B.00.00 Posted

At 9:00 AM Friday March 31th, 2017 we completed posting CHT Build Release 21B.00.00.

Study the information provided on our March What's New Page to find out what new and changed features are available in this build. Further details will appear in this, our April 2017 What's New Page as the month progresses.

We will also mail, as usual, our Build 21B.00.00 Facts Sheet by mid-week, April 5th, 2017.

Gus Creces
The Clarion Handy Tools Page

www.cwhandy.ca

gcreces@gmail.com

April 1, 2017

 


The Latest Docs

(Regenerated Monthly)

The latest template documents (HTML) are here:

CHT Template Docs

The latest demo application docs are here:

CHT Application Docs (Complete)

The latest utility application docs are here:

CHT Utility Applications Docs

The latest BATCH-BOT application docs are here:

CHT Batch-Bot Application Docs

The latest SNAP-IN application docs are here:

CHT Snap-In Application Docs

The latest classes docs are here:

CHT Classes Docs