title,body,link,tags,view_count,is_answered,score,answer_count
Android OS stablity/Stress testing,"<p>I am currently working for Mobile OEM, our mobiles are based on Android. I want to test stability and stress the Android OS of the mobile. Currently, we Do soak testing, monkey test. I am wondering if there is any other way to test the stability of the Android OS</p>
",https://stackoverflow.com/questions/55039679/android-os-stablity-stress-testing,"['android', 'testing']",58,False,-2,1
django handle slowly when doing Stress Testing,"<p>i had built django server to handle some work.And when i sent 10000 requests to django,django will became slow after 4000th requests,it will report socket timeout randomly.And i take this one data to test it alone,it is ok.So,how can i fix this bug?</p>

<pre><code>i have no idea to fix it

def handle(request) :
  sys.stderr.write( ""[info]request come in {} \n"".format(sys._getframe().f_code.co_name))

  ret_dict = dict()
  ret_str = """"
  if (""vid"" in request.GET) and (""title"" in request.GET):
    id = request.GET[""id""]
    title = urllib.parse.unquote(request.GET[""title""],encoding ='utf-8')
    title = request.GET[""title""]

    req_video = dict()
    req_video[""vid""] =  vid
    req_video[""title""] = title
    tags = handle_function(req_video)
    ret_dict[""vid""] = vid
    ret_dict[""tags""] = tags
    ret_str = json.dumps(ret_dict)


  return HttpResponse(ret_str)
</code></pre>

<p>client codes:</p>

<pre><code>for line in sys.stdin:
            line = line.rstrip('\n')
            fields = line.split(""\t"")
            vid = fields[0]
            title = fields[1]
            page_params_mq = {'vid':vid,'title':title}
            try :
                    rec_mq_res = get_rec_mingqiang(server_ip_mq, server_port_mq, web_page_mq, page_params_mq)
            except:
                    print (line)
                    continue
            vid = rec_mq_res[""vid""]
            tags = rec_mq_res[""tags""]
            fo.write(vid+""\t""+tags+""\n"")
</code></pre>

<p>cat test_data|python client.py, there are 10000 rows in test_data,and it send request sequentially,and will be hunt randomly after 4000.
django can handle these requests quickly</p>
",https://stackoverflow.com/questions/57428290/django-handle-slowly-when-doing-stress-testing,['django'],132,False,1,0
Stress testing Azure App Service periodically stops processing requests,"<p>I am currently stress testing a .Net Core application, targeting netcoreapp2.2, that is hosted on Azure as a App Service connected to a P1V2 (210 ACU, 3.5GB memory) service plan with 2 instances.</p>

<p>The endpoint that I'm stress testing is very simple, it validates a Oauth2.0 token, gets the user and some info about the user from a P2 (250 DTU) Azure hosted database, total 4 db queries per request, and returns the string ""Pong"".</p>

<p>When running 15 concurrent users (or more) in 200 loops I see the stop(s) in processing seen in the image (between the high peaks). The service plan never hits more than around 20-35% CPU and the database never uses more than 2% load. Increasing the users decreases the average throughput.</p>

<p>When looking at the slow requests it is like it just randomly stops, never at the same place. When I look at the DB requests I never see a request that takes longer than a couple of 100 milliseconds while some requests can take upwards to 5-6s to process.</p>

<p>It feels like I reach some limit which results in something stopping for a period of time, but I can't figure out where the problem lies.</p>

<p>When running the same stress locally I don't see these stops.</p>

<p>I'm using jmeter cli to run the stress tests against both environments.</p>

<p>Any help is greatly appreciated, thanks!</p>

<p><a href=""https://i.stack.imgur.com/A31jy.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/A31jy.png"" alt=""enter image description here""></a></p>
",https://stackoverflow.com/questions/54956650/stress-testing-azure-app-service-periodically-stops-processing-requests,"['azure', '.net-core', 'azure-sql-database', 'stress-testing', 'azure-app-service-plans']",279,True,3,1
What is the duration of Stress Testing in Apache JMeter?,"<p>I had tried doing stress testing in jmeter but could not get the timing in which it should be finished</p>
",https://stackoverflow.com/questions/55760807/what-is-the-duration-of-stress-testing-in-apache-jmeter,['jmeter'],217,False,0,2
"C Programming :: accept() works and then doesn&#39;t, in stress-testing","<p>I am coding what I thought would be a simple C socket program.  The code will act as a primitive server.  It needs to create a listening socket, and then each time it is contacted, it needs to <strong>accept()</strong> the incoming connection, then spin off a thread to handle some basic processing.  The threading and processing part I more-or-less have working, but the <strong>accept()</strong> part breaks down after the code successfully handles some initial connections..  This is a problem, because I anticipate that this program will need to run continuously and may have to process a lot of incoming requests.</p>

<p>Some environmental stuff first:  I’m working on a Ubuntu box, and my code is compiled with GCC:</p>

<pre><code>root@ubuntu:/home/me/socketProject# gcc -v
...
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
root@ubuntu:/home/me/socketProject#
</code></pre>

<p>My <strong>main()</strong> function sets up a listening socket, then starts an infinite loop.  For every connection that comes in, it tries to <strong>accept()</strong> and then pass the work off to a detached thread:</p>

<pre><code>int main(int argc, char ** argv)
{
   int             sock = -1;
   struct          sockaddr_in address;
   int             port = 12345;
   connection_t*   connection;
   pthread_t       thread;

   // Create the listening socket
   sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
   if (sock &lt;= 0){
      fprintf(stderr, ""%s: error: cannot create socket\n"", argv[0]);
      return -3;
   }

   // Bind socket to port
   address.sin_family = AF_INET;
   address.sin_addr.s_addr = INADDR_ANY;
   address.sin_port = htons(port);
   if (bind(sock, (struct sockaddr *)&amp;address, sizeof(struct sockaddr_in)) &lt; 0){
      fprintf(stderr, ""%s: error: cannot bind socket to port %d\n"", argv[0], port);
      return -4;
   }

   // Listen on the port
   if (listen(sock, 5) &lt; 0){
      fprintf(stderr, ""%s: error: cannot listen on port\n"", argv[0]);
      return -5;
   }

   // Listen forever...
   while (1){
      // Accept incoming connections
      connection = (connection_t *)malloc(sizeof(connection_t));

      printf(""1)  [[ sock: %d -- address: %s -- addr_len: %d ]]\n"", connection-&gt;sock, &amp;connection-&gt;address.sa_data, connection-&gt;addr_len );

      connection-&gt;sock = accept(sock, &amp;connection-&gt;address, &amp;connection-&gt;addr_len);

      printf(""2)  [[ sock: %d -- address: %s -- addr_len: %d ]]\n"", connection-&gt;sock, &amp;connection-&gt;address.sa_data, connection-&gt;addr_len );

      if (connection-&gt;sock &lt;= 0){
         printf("" *** PROBLEM!!!  Value of errno: %d\n "", errno);
         free(connection);
      }
      else{
         // Start a new thread but do not wait for it
         pthread_create(&amp;thread, 0, process, (void *)connection);  // process() omitted
         pthread_detach(thread);
      }
   }
   return 0;
}
</code></pre>

<p>Pretty textbook stuff.  I worked out all the logic and got everything working perfectly when there is only one incoming connection.</p>

<p>But then I started stress-testing, and immediately realized I had a problem.  I wrote a client program that send X number of test requests, all in a row.  The above server code works fine for a few initial requests, but then breaks down.  Here’s the output:</p>

<pre><code>root@ubuntu:/home/me/socketProject# ./server
        *** 1)  [[ sock: 18954128 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 7 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18970816 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 8 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18962080 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 9 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18962112 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 10 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18954064 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 11 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18954160 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 12 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18907120 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 13 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18950880 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 14 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 15 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18957136 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 17 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18957168 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
</code></pre>

<p>You see the problem.  In the above example, something happens after about ten successful requests, then <strong>accept()</strong> starts complaining of <strong>ERRNO 22</strong>.  A quick Google search revealed that <strong>ERRNO 22</strong> is “<strong><em>EINVAL: Invalid argument</em></strong>”, which obviously isn’t good.</p>

<p>I’ve done some simple socket programming from my student days, but I’ve never had a problem with <strong>accept()</strong> before.  I’m really puzzled.  It looks to me like N-1 requests arrive and are successfully processed by <strong>accept()</strong>.  But then the Nth appears, <strong>accept()</strong> doesn’t like it, and the socket gets corrupted.  How or why, I don’t know.</p>

<p><strong>accept()</strong> is complaining about ""invalid argument(s)"".  But the client is literally sending the exact same test message over and over.  How can the N-1 connections present valid arguments, but the Nth, N+1th, N+2th... argument be invalid?  That blows my mind.</p>

<p>Some observations:</p>

<ul>
<li><p>This is a black-and-white phenomenon.  By that, I mean that the
socket works perfectly 100% of the time, but then something bad
happens with <strong>accept()</strong>, and then the socket never works again.</p></li>
<li><p>I’ve run a lot of tests, and there’s no telling how many successful
connections I’ll get when I start a new test.  I’ve had as many as 88
successful connections before the socket stops working.  In another
test, I only had 2 successful connections before the socket went
belly-up.  The average number of successful connections before the
socket breaks is about 17.</p></li>
<li><p>My client test program does not try to space out the requests it
sends; it simply opens a socket and starts bombarding my server
program at top speed.  I’m doing this because I want to stress-test. 
But perhaps my client program is sending test requests too fast for
the server to keep up…?</p></li>
<li><p>It is also possible, I suppose, that the client program is causing
the problem.  I don’t think so, because it just sends the same test
message over and over and it never throws an error.</p></li>
</ul>

<p>Has anyone ever seen an issue like this?  I’ve been looking on SO for about an hour, but I can’t find another post where <strong>accept()</strong> works, and then doesn’t.</p>

<p>Any help/advice is appreciated.</p>
",https://stackoverflow.com/questions/57981377/c-programming-accept-works-and-then-doesnt-in-stress-testing,"['c', 'sockets', 'stress-testing']",140,True,0,1
What is the duration of Stress Testing in Apache JMeter?,"<p>I had tried doing stress testing in jmeter but could not get the timing in which it should be finished</p>
",https://stackoverflow.com/questions/55760807/what-is-the-duration-of-stress-testing-in-apache-jmeter,['jmeter'],217,False,0,2
How can I make apache ignite use all CPU for SQL query?,"<p>I'm doing SQL Stress Testing for Apache Ingite 2.7.6 on one node.I can not make cpu of server up to 100%,it always keep stable when cup up to about 80%,No matter how many requests you add.
config is:</p>

<pre><code>&lt;beans xmlns=""http://www.springframework.org/schema/beans""
      xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance""
      xsi:schemaLocation=""
      http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans.xsd""&gt;
   &lt;!--
                Alter configuration below as needed.
   --&gt;
&lt;bean id=""grid.cfg"" class=""org.apache.ignite.configuration.IgniteConfiguration""&gt;
   &lt;property name=""clientMode"" value=""false""/&gt;
   &lt;!-- Explicitly enable peer class loading. --&gt;
   &lt;property name=""peerClassLoadingEnabled"" value=""true""/&gt;
     &lt;!-- Consistent globally unique node identifier which survives node restarts. --&gt;
     &lt;!-- Configure internal thread pool. --&gt;
      &lt;property name=""discoverySpi""&gt; 
            &lt;bean id=""discoverySpi"" class=""org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi""&gt;

               &lt;!-- Initial local port to listen to. --&gt;
           &lt;property name=""localPort"" value=""47500""/&gt;  
           &lt;!-- Changing local port range. This is an optional action. --&gt;
           &lt;property name=""localPortRange"" value=""1""/&gt;  

           &lt;!-- Setting up IP finder for this cluster --&gt;
           &lt;property name=""ipFinder""&gt;        
               &lt;bean class=""org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder""&gt;
                   &lt;property name=""addresses""&gt;
                       &lt;list&gt;
                           &lt;value&gt;10.101.1.41&lt;/value&gt;
                       &lt;/list&gt;
                   &lt;/property&gt;
               &lt;/bean&gt;
           &lt;/property&gt;
           &lt;/bean&gt;
      &lt;/property&gt;

     &lt;property name=""longQueryWarningTimeout"" value=""20""/&gt;
     &lt;property name=""queryThreadPoolSize"" value=""16""/&gt;
   &lt;property name=""publicThreadPoolSize"" value=""16""/&gt;
   &lt;property name=""systemThreadPoolSize"" value=""16""/&gt;
   &lt;property name=""serviceThreadPoolSize"" value=""16""/&gt;
   &lt;property name=""stripedPoolSize"" value=""16""/&gt;
    &lt;property name=""dataStreamerThreadPoolSize"" value=""16""/&gt;
    &lt;!-- property name=""rebalanceThreadPoolSize"" value=""1""/ --&gt;
   &lt;property name=""authenticationEnabled"" value=""true""/&gt;

    &lt;property name=""clientConnectorConfiguration""&gt;
       &lt;bean class=""org.apache.ignite.configuration.ClientConnectorConfiguration""&gt;
           &lt;property name=""host"" value=""10.101.1.41""/&gt;
           &lt;property name=""port"" value=""12800""/&gt;
           &lt;property name=""portRange"" value=""1000""/&gt;
           &lt;property name=""maxOpenCursorsPerConnection"" value=""2000""/&gt;
                 &lt;property name=""socketSendBufferSize"" value=""65536""/&gt;
                 &lt;property name=""socketReceiveBufferSize"" value=""131072""/&gt;
                 &lt;property name=""threadPoolSize"" value=""16""/&gt;
                   &lt;/bean&gt;
   &lt;/property&gt;
       &lt;property name=""consistentId"" value=""TEST1""/&gt;
       &lt;property name=""dataStorageConfiguration""&gt;
           &lt;bean class=""org.apache.ignite.configuration.DataStorageConfiguration""&gt;
               &lt;property name=""walSegmentSize"" value=""#{1L * 1024 * 1024 * 1024}""/&gt;
          &lt;property name=""writeThrottlingEnabled"" value=""true""/&gt;
                 &lt;property name=""concurrencyLevel"" value=""8""/&gt;
                 &lt;property name=""pageSize"" value=""8192""/&gt;
                   &lt;property name=""walCompactionEnabled"" value=""true"" /&gt;
         &lt;property name=""walCompactionLevel"" value=""6"" /&gt;

               &lt;!-- Redefining the default region's settings --&gt;
               &lt;property name=""defaultDataRegionConfiguration""&gt;
                   &lt;bean class=""org.apache.ignite.configuration.DataRegionConfiguration""&gt;
                               &lt;property name=""name"" value=""20GB_Region""/&gt;
                           &lt;property name=""initialSize"" value=""#{500L * 1024 * 1024}""/&gt;
                       &lt;property name=""maxSize"" value=""#{10L * 1024 * 1024 * 1024}""/&gt;
                       &lt;property name=""persistenceEnabled"" value=""true""/&gt;
                      &lt;property name=""checkpointPageBufferSize""  value=""#{4 * 1024 * 1024 * 1024}""/&gt;   
                      &lt;property name=""evictionThreshold"" value=""0.95"" /&gt;
                       &lt;property name=""metricsEnabled"" value=""true""/&gt;
                       &lt;property name=""metricsRateTimeInterval"" value=""10000""/&gt;
                       &lt;property name=""metricsSubIntervalCount"" value=""1024""/&gt;   
                   &lt;/bean&gt;
               &lt;/property&gt;
               &lt;!-- &lt;property name=""storagePath"" value=""D:\\Test\\db"" /&gt; --&gt;
               &lt;property name=""storagePath"" value=""/home/igniteTT/ignite/data/db"" /&gt;
               &lt;!-- &lt;property name=""walPath"" value=""D:\\Test\\db\\wal"" /&gt; --&gt;
               &lt;property name=""walPath"" value=""/home/igniteTT/ignite/data/wal"" /&gt;
               &lt;!-- &lt;property name=""walArchivePath"" value=""D:\\Test\\db\\wal\\archive"" /&gt; --&gt;
               &lt;property name=""walArchivePath"" value=""/home/igniteTT/ignite/data/archive"" /&gt;
                &lt;property name=""walMode"" value=""LOG_ONLY""/&gt;
           &lt;/bean&gt;
       &lt;/property&gt;

   &lt;property name=""odbcConfiguration""&gt;
     &lt;bean class=""org.apache.ignite.configuration.OdbcConfiguration""&gt;
                 &lt;property name=""maxOpenCursors"" value=""512""/&gt;
                 &lt;property name=""threadPoolSize"" value=""16""/&gt;  
     &lt;/bean&gt;
   &lt;/property&gt;

   &lt;property name=""cacheConfiguration""&gt;
      &lt;list&gt;
              &lt;bean class=""org.apache.ignite.configuration.CacheConfiguration""&gt;
                     &lt;property name=""name"" value=""PUBLIC""/&gt;
                     &lt;property name=""cacheMode"" value=""PARTITIONED""/&gt;
                     &lt;property name=""atomicityMode"" value=""TRANSACTIONAL""/&gt;
                     &lt;property name=""writeSynchronizationMode"" value=""FULL_ASYNC""/&gt;

                    &lt;property name=""onheapCacheEnabled"" value=""true""/&gt;
                          &lt;property name=""evictionPolicy""&gt;
                               &lt;!-- LRU eviction policy. --&gt;
                               &lt;bean class=""org.apache.ignite.cache.eviction.lru.LruEvictionPolicy""&gt;
                                 &lt;property name=""maxSize"" value=""1000000""/&gt;
                               &lt;/bean&gt;
                          &lt;/property&gt;
                   &lt;property name=""sqlOnheapCacheEnabled"" value=""true""/&gt;
                   &lt;property name=""queryParallelism"" value=""16""/&gt;
                   &lt;property name=""maxQueryIteratorsCount"" value=""4096""/&gt;
                    &lt;property name=""maxConcurrentAsyncOperations"" value=""1000""/&gt;
                   &lt;property name=""queryDetailMetricsSize"" value=""0""/&gt;
                 &lt;property name=""rebalanceMode"" value=""SYNC""/&gt;
                 &lt;property name=""rebalanceBatchSize"" value=""#{2 * 1024 * 1024}""/&gt;
                 &lt;!-- Set throttle interval. --&gt;
                 &lt;property name=""rebalanceThrottle"" value=""100""/&gt;
                 &lt;property name=""dataRegionName"" value=""20GB_Region""/&gt;

                     &lt;property name=""queryEntities""&gt;
                       &lt;list&gt;
                         &lt;bean class=""org.apache.ignite.cache.QueryEntity""&gt;
                           &lt;property name=""keyType"" value=""java.lang.Long""/&gt;
                           &lt;property name=""keyFieldName"" value=""BAL_ID""/&gt;
                           &lt;property name=""valueType"" value=""BAL""/&gt;
                           &lt;property name=""fields""&gt;
                             &lt;map&gt;
                               &lt;entry key=""BAL_ID"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""ACCT_ID"" value=""java.lang.Integer""/&gt;
                               &lt;entry key=""ACCT_RES_ID"" value=""java.lang.Integer""/&gt;
                             &lt;entry key=""EFF_DATE"" value=""java.sql.Timestamp""/&gt;
                             &lt;entry key=""EXP_DATE"" value=""java.sql.Timestamp""/&gt;
                             &lt;entry key=""UPDATE_DATE"" value=""java.sql.Timestamp""/&gt;
                               &lt;entry key=""GROSS_BAL"" value=""java.lang.Long""/&gt;       
                               &lt;entry key=""RESERVE_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""CONSUME_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""RATING_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""BILLING_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""LAST_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""INIT_BAL"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""CEIL_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""DAILY_CEIL_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""VAR_CEIL_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""FLOOR_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""DAILY_FLOOR_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""VAR_CELL_LIMIT"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""PRIORITY"" value=""java.lang.Integer""/&gt;
                               &lt;entry key=""BAL_USED""  value=""java.lang.Integer""/&gt;
                               &lt;entry key=""RESERVE_IN"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""RESERVE_PS""  value=""java.lang.Long""/&gt;
                               &lt;entry key=""LAST_RECHARGE"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""SUBS_ID""  value=""java.lang.Long""/&gt;
                               &lt;entry key=""IS_USED"" value=""java.lang.Long""/&gt;
                               &lt;entry key=""RATIO""  value=""java.lang.Long""/&gt;
                             &lt;/map&gt;
                           &lt;/property&gt;

                           &lt;property name=""indexes""&gt;
                               &lt;list&gt;
                                   &lt;bean class=""org.apache.ignite.cache.QueryIndex""&gt;
                                       &lt;constructor-arg value=""BAL_ID""/&gt;
                                   &lt;/bean&gt;
                               &lt;/list&gt;
                           &lt;/property&gt;
                         &lt;/bean&gt;
                       &lt;/list&gt;
                     &lt;/property&gt;
           &lt;/bean&gt;
      &lt;/list&gt;
   &lt;/property&gt;
 &lt;property name=""communicationSpi""&gt;
   &lt;bean class=""org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi""&gt;
       &lt;property name=""localPort"" value=""48100""/&gt;
       &lt;property name=""ackSendThreshold"" value=""512""/&gt;
     &lt;property name=""slowClientQueueLimit"" value=""50000000""/&gt;
   &lt;/bean&gt;
 &lt;/property&gt;
&lt;/bean&gt;
&lt;/beans&gt;
</code></pre>

<p>and Server Metrics is</p>

<pre><code>[16:16:20,931][INFO][grid-timeout-worker-#39][IgniteKernal] 
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
    ^-- Node [id=6ba8f92a, uptime=00:06:00.051]
    ^-- H/N/C [hosts=1, nodes=1, CPUs=8]
    ^-- CPU [cur=73.4%, avg=8.9%, GC=1.5%]
    ^-- PageMemory [pages=90]
    ^-- Heap [used=2878MB, free=59.4%, comm=3300MB]
    ^-- Off-heap [used=0MB, free=99.99%, comm=10540MB]
    ^--   sysMemPlc region [used=0MB, free=99.96%, comm=100MB]
    ^--   metastoreMemPlc region [used=0MB, free=99.92%, comm=100MB]
    ^--   20GB_Region region [used=0MB, free=99.99%, comm=10240MB]
    ^--   TxLog region [used=0MB, free=100%, comm=100MB]
    ^-- Ignite persistence [used=0MB]
    ^--   sysMemPlc region [used=0MB]
    ^--   metastoreMemPlc region [used=unknown]
    ^--   20GB_Region region [used=0MB]
    ^--   TxLog region [used=0MB]
    ^-- Outbound messages queue [size=0]
    ^-- Public thread pool [active=0, idle=0, qSize=0]
    ^-- System thread pool [active=0, idle=6, qSize=0]
</code></pre>

<p>are there some configurations controlling the max usage of cpu??
 Help much appreciated,</p>
",https://stackoverflow.com/questions/58538561/how-can-i-make-apache-ignite-use-all-cpu-for-sql-query,"['sql', 'cpu', 'ignite']",242,True,0,1
UWP Controlled CpuBurner,"<p>I'm currently in a need to develop CPU stress testing functionality with UWP application. It will later be integrated into main application, currently developing it in a dedicated test ""app"". Basically I need functionality to set cpu usage to 100% and back to normal when user requests. Currently Event handlers can be simple button_clock handlers, like this:</p>

<pre><code>private void StawpButton_Click(object sender, RoutedEventArgs e)
    {
        //should stop stress-testing somehow...
    }
</code></pre>

<p>For burning cores I used <a href=""https://stackoverflow.com/questions/49203825/how-to-achieve-100-cpu-load-with-c-sharp-code"">This nifty code</a> which uses while(true loop) for stress-testing cpu. However this is rather fire-and-forget method and I dont know how to shut this kond of process down.</p>

<p>My Current Code looks like this:</p>

<pre><code>class CpuStresser
{


    public CpuStresser()
    {
        //do something when initializing
    }
    public static void DoStress()
    {
        for (int i = 0; i &lt; 32; i++)
        {
            Task.Run(() =&gt; KillCore());
        }

    }


    public static Task KillCore()
    {
        Random rand = new Random();
        long num = 0;
        while (true)
        {
            num += rand.Next(100, 1000);
            if (num &gt; 1000000) { num = 0; }

        }

    }

}
</code></pre>

<p>This works well for putting cpu to 100% load, but I'm unable to shut down tasks. What would be the best way to do it using event handlers?</p>

<p>I tried to add condition to while(true) loop, but error ""not all code paths return a value"" comes quick. Also making variable that is accessible to both KillCore task and EventHandler is rather difficult, because these are located in different classes.</p>
",https://stackoverflow.com/questions/54090798/uwp-controlled-cpuburner,"['uwp', 'task', 'cpu', 'stress-testing']",39,False,0,1
Stress testing Azure App Service periodically stops processing requests,"<p>I am currently stress testing a .Net Core application, targeting netcoreapp2.2, that is hosted on Azure as a App Service connected to a P1V2 (210 ACU, 3.5GB memory) service plan with 2 instances.</p>

<p>The endpoint that I'm stress testing is very simple, it validates a Oauth2.0 token, gets the user and some info about the user from a P2 (250 DTU) Azure hosted database, total 4 db queries per request, and returns the string ""Pong"".</p>

<p>When running 15 concurrent users (or more) in 200 loops I see the stop(s) in processing seen in the image (between the high peaks). The service plan never hits more than around 20-35% CPU and the database never uses more than 2% load. Increasing the users decreases the average throughput.</p>

<p>When looking at the slow requests it is like it just randomly stops, never at the same place. When I look at the DB requests I never see a request that takes longer than a couple of 100 milliseconds while some requests can take upwards to 5-6s to process.</p>

<p>It feels like I reach some limit which results in something stopping for a period of time, but I can't figure out where the problem lies.</p>

<p>When running the same stress locally I don't see these stops.</p>

<p>I'm using jmeter cli to run the stress tests against both environments.</p>

<p>Any help is greatly appreciated, thanks!</p>

<p><a href=""https://i.stack.imgur.com/A31jy.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/A31jy.png"" alt=""enter image description here""></a></p>
",https://stackoverflow.com/questions/54956650/stress-testing-azure-app-service-periodically-stops-processing-requests,"['azure', '.net-core', 'azure-sql-database', 'stress-testing', 'azure-app-service-plans']",279,True,3,1
Knowing how much one can load in three.js,"<p>So I was looking into loading character models, and a bit of stress testing by loading that same model several times to create an army. Worked fine on a high-end computer, but when testing it on a 32-bit work laptop it gave an Out Of Memory Error. My question is is there any way to know how much memory a model will take, whether it is external files or generated on runtime, then find out how much the client can handle, and moderate the memory usage appropriately (smaller army, load low-end models, reduce texture size, etc)?</p>
",https://stackoverflow.com/questions/54532468/knowing-how-much-one-can-load-in-three-js,"['memory', 'three.js']",85,True,0,1
How to make the successfully running pods delete themselves within the time set by the user,"<p>I am working on a new scheduler's stress test in Kubernetes. I need to open a lot of CPU and memory pods to analyze performance.</p>

<p>I am using image: <code>polinux/stress</code> in my pods.
I would like to ask if there is any instruction, or when I write the yaml file, I can set this successfully generated pod to delete itself within the time set by me.</p>

<p>The following yaml file is the pod I am writing for stress testing. I would like to ask if I can write it from here to let him delete it after a period of time.</p>

<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: alltest12
  namespace: test
spec:
  containers:
  - name: alltest
    image: polinux/stress
    resources:
      requests:
        memory: ""1000Mi""
        cpu: ""1""
      limits:
        memory: ""1000Mi""
        cpu: ""1""
    command: [""stress""]
    args: [""--vm"", ""1"", ""--vm-bytes"", ""500M"", ""--vm-hang"", ""1""]
</code></pre>
",https://stackoverflow.com/questions/56194426/how-to-make-the-successfully-running-pods-delete-themselves-within-the-time-set,['kubernetes'],78,False,0,1
"Allocating Java 12.0.1 memory space using -Xmx not working, -d64 not recognized","<p>I am currently running a GCP server utilizing Debian 9 and Java 12.0.1. I've been trying to allocate 7GB of RAM to the java server using <code>-Xms1G -Xmx7G</code> on startup however when stress testing the server and running <code>free -m</code> in the background, I haven't seen Java utilizing more than exactly 1.75GB of RAM, even when under immense stress.</p>

<p>I was previously using <code>-d64</code> to specify 64-bit Java with my old version of Java (openJDK 1.8 I believe?), however since upgrading to Java 12.0.1, I get an error when trying to use <code>-d64</code>, <code>Unrecognized option: -d64</code>.</p>

<p>My current VM has 7.5GB of RAM. Running <code>uname -a</code> gave me the following:</p>

<p><code>Linux server-name 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux</code></p>

<p><strong>Edit:</strong> Following up on a comment, see results of <code>ulimit -a</code> below.
<a href=""https://i.stack.imgur.com/3g4Su.jpg"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/3g4Su.jpg"" alt=""enter image description here""></a></p>
",https://stackoverflow.com/questions/56247423/allocating-java-12-0-1-memory-space-using-xmx-not-working-d64-not-recognized,"['java', 'google-cloud-platform', 'debian', 'java-12']",242,False,2,0
Apache Ignite OutOfMemory on low usage,"<p>I have deployed a StatefulSet of Apache Ignite in multiple testing Kubernetes Cluster. </p>

<p>I have passed my stress testing phase with my current configuration. However, I found that there are some OutOfMemory ERROR in Apache Ignite some in some new testing Cluster with much lower of load.</p>

<p>Below is a snapshot of log i extracted from 1 of the Apache Ignite instance:</p>

<pre><code>INFO: TCP discovery spawning a new thread for connection [rmtAddr=/10.254.174.226, rmtPort=45453]
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Runtime error caught during grid runnable execution: GridWorker [name=tcp-disco-client-message-worker, igniteInstanceName=null, finished=false, heartbeatTs=1573779638619, hashCode=373238347, interrupted=true, runner=tcp-disco-client-message-worker-#109]
java.lang.OutOfMemoryError: Java heap space

Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger error
SEVERE: Runtime error caught during grid runnable execution: IgniteSpiThread [name=tcp-disco-client-message-worker-#109]
java.lang.OutOfMemoryError: Java heap space
Exception in thread ""tcp-disco-client-message-worker-#109"" java.lang.OutOfMemoryError: Java heap space
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger info
INFO: TCP discovery accepted incoming connection [rmtAddr=/10.254.183.232, rmtPort=41313]
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger info
INFO: TCP discovery spawning a new thread for connection [rmtAddr=/10.254.183.232, rmtPort=41313]
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger info
INFO: Started serving remote node connection [rmtAddr=/10.254.174.226:45453, rmtPort=45453]
Nov 15, 2019 @ 09:01:26.612 Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger warning
Nov 15, 2019 @ 09:01:26.612 WARNING: New next node has connection to it's previous, trying previous again. [next=TcpDiscoveryNode [id=5cbb5f1c-ca74-4b2f-ba70-314f621ab997, addrs=[10.254.168.12, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, ignite-sit-5.ignite-sit.sit.svc.cluster.local/10.254.168.12:47500], discPort=47500, order=3922, intOrder=2000, lastExchangeTime=1573779246139, loc=false, ver=2.7.5#20190603-sha1:be4f2a15, isClient=false]]
Nov 15, 2019 @ 09:01:26.612 Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger info
Nov 15, 2019 @ 09:01:26.612 INFO: New next node [newNext=TcpDiscoveryNode [id=6fcccf11-f903-4b4a-bbac-730ca0b80ce8, addrs=[10.254.169.217, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, ignite-sit-4.ignite-sit.sit.svc.cluster.local/10.254.169.217:47500], discPort=47500, order=3912, intOrder=1993, lastExchangeTime=1573779190075, loc=false, ver=2.7.5#20190603-sha1:be4f2a15, isClient=false]]
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger info
INFO: Finished serving remote node connection [rmtAddr=/10.254.174.226:45453, rmtPort=45453
Nov 15, 2019 1:01:26 AM org.apache.ignite.logger.java.JavaLogger error
</code></pre>

<p>Sorry for the bad log formatting.</p>

<p>I would like to know what causes the OutOfMemory error and how can I prevent this from happening again.</p>

<p>Help would be much appreciated.</p>

<p>Update: 
Heapdump analysis result:</p>

<pre><code>The thread org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader @ 0xd9fbe2d0 tcp-disco-sock-reader-#369 keeps local variables with total size 312,295,344 (48.95%) bytes.
</code></pre>

<p>It looks like TCP SocketReader requires a lot of heap memory.</p>
",https://stackoverflow.com/questions/58907915/apache-ignite-outofmemory-on-low-usage,"['java', 'kubernetes', 'out-of-memory', 'ignite']",589,False,0,2
How to get CPU counters from stress-ng --perf?,"<p>I'm doing some stress testing on my CPUs, I'm using stress-ng as my stressor and I want to access my CPU cycles and Instructions. I use the flag --perf to get stress-ng to use perf events to measure information about the CPU. As you can see below I don't have access to CPU counters, like instruction calls and cache misses. I think it has something to do with admin access to the kernel. I am not really sure, is there someone who knows what I should do to make it work like the example at the bottom? <br />
<strong>What I get:</strong></p>

<pre><code>~$ sudo stress-ng --cpu 1 --perf --metrics-brief -t 10s
stress-ng: info:  [2611] dispatching hogs: 1 cpu
stress-ng: info:  [2611] successful run completed in 10.06s
stress-ng: info:  [2611] stressor       bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [2611]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [2611] cpu                2537     10.06      9.94      0.05       252.25       253.95
stress-ng: info:  [2611] cpu:
stress-ng: info:  [2611]                        746 Page Faults User               74,16 /sec 
stress-ng: info:  [2611]                          1 Page Faults Kernel              0,10 /sec 
stress-ng: info:  [2611]                         76 System Call Enter               7,56 /sec 
stress-ng: info:  [2611]                         75 System Call Exit                7,46 /sec 
stress-ng: info:  [2611]                          5 TLB Flushes                     0,50 /sec 
stress-ng: info:  [2611]                         20 Kmalloc                         1,99 /sec 
stress-ng: info:  [2611]                          1 Kmalloc Node                    0,10 /sec 
stress-ng: info:  [2611]                         94 Kfree                           9,34 /sec 
stress-ng: info:  [2611]                        125 Kmem Cache Alloc               12,43 /sec 
stress-ng: info:  [2611]                          1 Kmem Cache Alloc Node           0,10 /sec 
stress-ng: info:  [2611]                        188 Kmem Cache Free                18,69 /sec 
stress-ng: info:  [2611]                        696 MM Page Alloc                  69,19 /sec 
stress-ng: info:  [2611]                          2 MM Page Free                    0,20 /sec 
stress-ng: info:  [2611]                    5 938 RCU Utilization               590,32 /sec 
stress-ng: info:  [2611]                         24 Sched Migrate Task              2,39 /sec 
stress-ng: info:  [2611]                          0 Sched Move NUMA                 0,00 /sec 
stress-ng: info:  [2611]                        196 Sched Wakeup                   19,49 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Exec                 0,00 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Exit                 0,00 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Fork                 0,00 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Free                 0,00 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Hang                 0,00 /sec 
stress-ng: info:  [2611]                          0 Sched Proc Wait                 0,00 /sec 
stress-ng: info:  [2611]                         44 Sched Switch                    4,37 /sec 
stress-ng: info:  [2611]                          0 Signal Generate                 0,00 /sec 
stress-ng: info:  [2611]                          1 Signal Deliver                  0,10 /sec 
stress-ng: info:  [2611]                        304 IRQ Entry                      30,22 /sec 
stress-ng: info:  [2611]                        304 IRQ Exit                       30,22 /sec 
stress-ng: info:  [2611]                    3 098 Soft IRQ Entry                307,98 /sec 
stress-ng: info:  [2611]                    3 098 Soft IRQ Exit                 307,98 /sec 
stress-ng: info:  [2611]                          0 Writeback Dirty Inode           0,00 /sec 
stress-ng: info:  [2611]                          0 Writeback Dirty Page            0,00 /sec 
stress-ng: info:  [2611]                          0 Migrate MM Pages                0,00 /sec 
stress-ng: info:  [2611]                         44 SKB Consume                     4,37 /sec 
stress-ng: info:  [2611]                          0 SKB Kfree                       0,00 /sec 
stress-ng: info:  [2611]                          0 IOMMU IO Page Fault             0,00 /sec 
stress-ng: info:  [2611]                          0 IOMMU Map                       0,00 /sec 
stress-ng: info:  [2611]                          0 IOMMU Unmap                     0,00 /sec 

</code></pre>

<p><strong>What I want:</strong></p>

<pre><code># stress-ng --matrix 40 -t 30s --times --perf --tz --metrics-brief
stress-ng: info:  [12734] dispatching hogs: 40 matrix
stress-ng: info:  [12734] cache allocate: default cache size: 22528K
stress-ng: info:  [12734] successful run completed in 30.66s
stress-ng: info:  [12734] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [12734]                          (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [12734] matrix         2057213     30.02   1187.60      1.55     68534.05      1729.99
stress-ng: info:  [12734] matrix:
stress-ng: info:  [12734]          2,418,977,858,680 CPU Cycles                    78.88 B/sec
stress-ng: info:  [12734]          3,616,346,898,160 Instructions                   0.12 T/sec (1.495 instr. per cycle)
stress-ng: info:  [12734]                216,338,520 Cache References               7.05 M/sec
stress-ng: info:  [12734]                 30,689,400 Cache Misses                   1.00 M/sec (14.19%)
stress-ng: info:  [12734]            457,662,770,480 Branch Instructions           14.92 B/sec
stress-ng: info:  [12734]              3,541,729,200 Branch Misses                  0.12 B/sec ( 0.77%)
stress-ng: info:  [12734]             29,326,176,200 Bus Cycles                     0.96 B/sec
stress-ng: info:  [12734]          2,463,405,104,320 Total Cycles                  80.33 B/sec
stress-ng: info:  [12734]                      6,240 Page Faults Minor            203.49 sec
stress-ng: info:  [12734]                          0 Page Faults Major              0.00 sec
stress-ng: info:  [12734]                    103,360 Context Switches               3.37 K/sec
stress-ng: info:  [12734]                      1,280 CPU Migrations                41.74 sec
stress-ng: info:  [12734]                          0 Alignment Faults               0.00 sec
stress-ng: info:  [12734]                      2,360 System Call Enter   
...
</code></pre>
",https://stackoverflow.com/questions/57960754/how-to-get-cpu-counters-from-stress-ng-perf,"['linux', 'cpu', 'ubuntu-18.04', 'stress-testing', 'perf']",761,False,0,0
Google App Engine - random ssl connection resets,"<p>I have a servlet hosted within Google App Engine. As part of the business logic, it performs a HTTPS request to another webservice. I started getting SSL connection reset exceptions but they appear to be happening in a random fashion. I haven't been able to repeat the exception and they start happening even more over time but then can fade away and come back. I've performed stress testing on the remote service directly from other machines and never receive a single issue. The only way I've been able to deal with this is by redeploying the application but it can take a few redeployments before the issue goes away. I am confident it isnt an issue with the remote service but something happening within Google App Engine. </p>
",https://stackoverflow.com/questions/57929827/google-app-engine-random-ssl-connection-resets,['google-app-engine'],49,True,2,1
Disable &quot;remember every SQL query&quot; (Django),"<p>I know this:</p>

<blockquote>
  <p>It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you’re debugging, but it’ll rapidly consume memory on a production server.</p>
</blockquote>

<p>Source: <a href=""https://docs.djangoproject.com/en/dev/ref/settings/#debug"" rel=""nofollow noreferrer"">https://docs.djangoproject.com/en/dev/ref/settings/#debug</a></p>

<p>I have a python script which does stress testing.</p>

<p>This script does <em>not</em> use the django test runner.</p>

<p>It uses the ORM in a loop doing more and more traffic in every iteration.</p>

<p>It would be great if my script could work in any environment. </p>

<p>I don't want other developers to force to fiddle with their settings (configure DEBUG  to False) before executing this script.</p>

<p>Is there a way to disable the particular feature ""remember every SQL query""?</p>
",https://stackoverflow.com/questions/55185241/disable-remember-every-sql-query-django,"['django', 'django-orm']",98,True,0,1
I have got an requirement for load testing . i was given both avg. and peak TPH. also avg and peak user load,"<p>I have to do performance testing of an ecommerce application where i got the details needed like Avg TPH and peak TPH . also Avg User and Peak User.</p>

<p>for e.g., an average of 1000 orders/hour, the peak of 3000 orders/hour during the holiday season, expected to grow to 6000 orders/hour next holiday season.</p>

<p>I was afraid which value to be considered for current users and TPH for performing load test for an hour.
also what load will be preferable foe stress testing and scalability testing. 
It would be a great helpful not only for the test point of view but also will help me in understanding the conceptually which would help me in a great deal down the lane.</p>
",https://stackoverflow.com/questions/57477627/i-have-got-an-requirement-for-load-testing-i-was-given-both-avg-and-peak-tph,"['jmeter', 'performance-testing', 'load-testing', 'loadrunner', 'stress-testing']",354,True,0,2
How to stop Trogdor testing Agent + Coordinator service responsible for shutting down Kafka and Zookeeper instances,"<p>I have been running stress testing on Kafka. I used trogdor test framework for creating network and IO faults. After testing I tried to kill process agents but the trogdor is running zombie processes which can't be killed by kill -9. 
The running trogdor service is killing zookeeper and Kafka instances and my cluster can't be active. What is the way to shutdown these service safely without the need to restart Unix server  ?
I would also appreciate a way to find parent process responsible for this. I currently have to use ps -o ppid= </p>

<p>I've tried killing Trogdor process, AgentClient process, agent and coordinator process</p>

<p>When I try to kill the parent process of any of the above mentioned processes, the connection to SSH server is disconnected and PuTTY is closed</p>
",https://stackoverflow.com/questions/58136740/how-to-stop-trogdor-testing-agent-coordinator-service-responsible-for-shutting,"['testing', 'apache-kafka', 'performance-testing']",174,False,1,1
Spring RestController 6 parallel execution?,"<p>I developing a SpringBoot-Application. During stress-testing the application shows some strange behaviour according parallel execution. AFAIK there should not be some parallel execution...</p>

<p>To test this behaviour i created this RestController:</p>

<pre><code>@RestController
public class TestController2 {

private static int instancecount = 0;

@PostConstruct
public void con() {
    System.out.println(""instancecount: "" + ++instancecount);
}

@PreDestroy
public void des() {
    System.out.println(""PreDestroy"");
}

@RequestMapping(value = { ""/"", ""/**"" }, method = RequestMethod.GET)
@ResponseBody
public ResponseEntity&lt;String&gt; entrypoint(@RequestHeader Map&lt;String, String&gt; requestHeaders) {
    System.out.println(""PRE!!!!!!!!!!!!!"");
    try {
         Thread.sleep(120000);
    } catch (InterruptedException e) {
    }
    System.out.println(""POST!!!!!!!!!!!!!!!!!"");

    return new ResponseEntity&lt;String&gt;(""qwertz"", null, HttpStatus.OK);
}

}
</code></pre>

<p>The application compiles and works like expected. Now i open the browser with 14 tabs simultaneously.</p>

<p>In the log i can see instant:</p>

<p>PRE!!!!!!!!!!!!!</p>

<p>after approximately 15 sek. the next 5:
PRE!!!!!!!!!!!!!
PRE!!!!!!!!!!!!!
PRE!!!!!!!!!!!!!
PRE!!!!!!!!!!!!!
PRE!!!!!!!!!!!!!</p>

<p>After 2 minutes the first finished. The other 5 finished at ~2:15.</p>

<p>This repeats until every 14 tabs are finished.</p>

<p>and now my question:</p>

<ol>
<li>Why is there a parallel execution?</li>
<li>Why 6?</li>
<li>Why the second starts after ~15 seconds?</li>
</ol>

<p>I am a little bit confused. I don't know where or what i have to look for?</p>

<p>Need input!</p>

<p>THX</p>
",https://stackoverflow.com/questions/58432195/spring-restcontroller-6-parallel-execution,"['spring-boot', 'spring-restcontroller']",260,True,1,1
How to continue to consume a memory after &quot;MemoryError&quot; was caught,"<p>I'm developing a tool for a stress testing. Its purpose is to consume all free memory. My problem is to constantly eat all free memory even if some amount of it was freed during script execution and especially after MemoryError was caught.</p>

<p>I tried to handle ""MemoryError"" exception with pass or continue statements, but it won't help. If MemoryError occurs script doesn't consume newly freed memory. Maybe it happens due to some OS limitations. </p>

<pre><code>def mem_cons(x):
timestamp()
a = []
idx = 0
appender = a.append
MEGA_STR = 'F' * (10 ** 4 * x)
try:
    while True:
        try:
            idx += 1
            if idx &gt; 10000:
                if 'kill' in flag:
                    print("""")
                    timestamp()
                    print(""Memory consumption was remotely stopped.\nPlease use \'ctrl+c\' command to exit"")
                    a = []
                    timestamp()
                    print(""Memory was cleared"")
                    break
                idx = 0
            appender(MEGA_STR)
        except MemoryError:
            continue
            timestamp()
except KeyboardInterrupt:
    print("""")
    timestamp()
    print(""Program has been stopped"")
</code></pre>

<p><strong>Scenario:</strong></p>

<ul>
<li>Script eats memory from 20% to 100%</li>
<li>MemoryException was caught</li>
<li>Some memory was freed manually lets say - 5.1%</li>
</ul>

<p><strong>Actual result:</strong></p>

<p>After that entire memory consumption level is still at 94.9% and doesn't growing (as we can see at <a href=""https://i.stack.imgur.com/H1Ymu.png"" rel=""nofollow noreferrer"">graph</a> - after the red line)</p>

<p><strong>Expected result:</strong></p>

<p>Script continues to consume manually freed memory and holds its level at 100% after MemoryException was caught.</p>

<p><strong>Thanks in advance!</strong></p>
",https://stackoverflow.com/questions/54557370/how-to-continue-to-consume-a-memory-after-memoryerror-was-caught,"['linux', 'python-2.7', 'out-of-memory']",29,False,1,0
How to increase Max connections WCF NET TCP,"<p>I'm setting the WCF NET TCP server. So far I'm going good with few connections.
But since I've decided to do some stress testing, I've found some problems.
With a client starting threads, I get 128 successful connections, but at the 129th connections, I get: </p>

<blockquote>
  <p>too busy servers exception.</p>
</blockquote>

<p>I've already changed maxConnections to 1000 at the binding. I've already changed maxConcurrentCalls, maxConcurrentSessions, maxConcurrentInstances to 5000 (testing only) at serviceThrottling.</p>

<p>My Binding:</p>

<pre><code>&lt;binding name=""netTcpClientsBindingConfig"" transferMode=""Buffered"" maxReceivedMessageSize=""50242880""
    closeTimeout=""00:05:00"" openTimeout=""00:05:00"" receiveTimeout=""00:05:00""
    sendTimeout=""00:05:00"" maxConnections=""1000""&gt;
</code></pre>

<p>My Behavior config:</p>

<pre><code>&lt;behavior name=""mexClientsBehavior""&gt;
    &lt;serviceThrottling
        maxConcurrentCalls     = ""5000""
        maxConcurrentSessions  = ""5000""
        maxConcurrentInstances = ""5000""
    /&gt;
</code></pre>

<p>I'm expecting to accept at least 1000 connections.</p>

<p>There is one limitation here (I believe so! Please can anyone correct me if there is another solution): </p>

<p>I need to retain the connection OPENED, because my server may contact the client through callbacks to send information that the clients should know about as soon as possible. If the connection is closed, I cannot re-open at the server side, or can I? That would be a solution.</p>
",https://stackoverflow.com/questions/56094502/how-to-increase-max-connections-wcf-net-tcp,"['c#', 'wcf']",713,True,0,1
Code is working fine in my system but coursera autograder is giving me unknown signal,"<p><strong>Task</strong> -- The goal in this code problem is to implement the binary search algorithm.</p>

<p><strong>Input Format</strong> -- The first line of the input contains an integer n and a sequence a0 &lt; a1 &lt; ... &lt; an−1 of n pairwise distinct positive integers in increasing order. The next line contains an integer k and k positive integers b0,b1,...,bk−1.</p>

<p><strong>Constraints</strong> -- 1 ≤ n,k ≤ 10^4; 1 ≤ a[i] ≤ 10^9 for all 0 ≤ i &lt; n; 1 ≤ b[]j ≤ 10^9 for all 0 ≤ j &lt; k;</p>

<p><strong>Output Format</strong> -- For all i from 0 to k−1, output an index 0 ≤ j ≤ n−1 such that aj = bi or −1 if there is no such index.</p>

<p>I am using code blocks with c++11 compiler.
I have already tried stress testing and got correct results in my system.
But coursera autograder is giving me unknown signal 11.
In some problems it gives unknown signal 8.
Can anyone tell me the possible reason behind this.</p>

<pre><code>int binary_search(const vector&lt;long long&gt; &amp;a, long long x) {
  size_t left = 0, right = (size_t)a.size()-1;
  size_t mid = 0;
  while(left&lt;=right){
    mid = (left+right)/2;
    if(x &lt; a[mid]){
        right = mid-1;
    }
    else if(x &gt; a[mid]){
        left = mid+1;
    }
    else return mid;
  }
  return -1;
}
int main() {
  size_t n;
  std::cin &gt;&gt; n;
  vector&lt;long long&gt; a(n);
  for (size_t i = 0; i &lt; a.size(); i++) {
    std::cin &gt;&gt; a[i];
  }
  size_t m;
  std::cin &gt;&gt; m;
  vector&lt;long long&gt; b(m);
  for (size_t i = 0; i &lt; m; ++i) {
    std::cin &gt;&gt; b[i];
  }
  for (size_t i = 0; i &lt; m; ++i) {
    //replace with the call to binary_search when implemented
    std::cout &lt;&lt; binary_search(a, b[i]) &lt;&lt; ' ';
  }
}
</code></pre>

<p>Actual result that i got in autograder.</p>

<pre><code>Failed case #4/36: unknown signal 11 (Time used: 0.00/1.00, memory used: 40071168/536870912.)
</code></pre>
",https://stackoverflow.com/questions/56472816/code-is-working-fine-in-my-system-but-coursera-autograder-is-giving-me-unknown-s,"['c++', 'algorithm', 'c++11', 'binary-search', 'coursera-api']",2309,True,3,3
JMeter fails on long scenario,"<p>I have a scenario with 5K HTTP requests. When I start JMeter with it, JMeter simply hangs after about 170 users. I followed all the guidelines for successful stress testing (no listeners, headless, increased heap space).</p>

<p>I must say that some of those requests are a little big, the overall file is ~115M. </p>

<p>When I only take a subset of the requests (~100), the simulation works better (faster initialization of users, holds more than 170 users, etc).</p>

<p>My question is, first, as I understand JMeter loads the scenario tree and every threads plays it, there should not be any duplication, so what exactly causes this extensive load? and second, what can I do about it?</p>

<p>PS: when I view the system bottlenecks I notice both CPU and memory are at very high values on the long file, both of the metrics have low values on the shorter version. Anyone can explain?</p>

<p>PS2: the requests have about 7 seconds of delay between them</p>
",https://stackoverflow.com/questions/55986898/jmeter-fails-on-long-scenario,['jmeter'],37,False,0,1
JMeter Load Tesing::Moving from WebDriver sampler To JSR223 sampler with groovy,"<p>I'm a newbie in jmeter so accept my apology for asking basics and long post.<br>
My Ultimate goal is to do Load and stress testing for a ajax enabled  application (IBM Maximo if any one cm across it)</p>

<p>Till now I was working in Jmeter WebDriver Sampler(selenium) and I was very comfortable with it and enjoying it thoroughly.   </p>

<p>Now I cm across this <a href=""https://www.blazemeter.com/blog/how-load-test-ajaxxhr-enabled-sites-jmeter"" rel=""nofollow noreferrer"">article</a>. So I decided to move on JSR223 Sampler and groovy (2nd approach as described in article)..
Now here is My questions List :)</p>

<ol>
<li>Is my decision is in right direction.should i move from Webdriver Sampler.  </li>
<li>Is JSR223 sampler is one solider army or it work with http Request Sampler.  </li>
<li>Is it able to handle Ajax request.  </li>
<li>Can any one share how a test plan structure looks like if it use JSR223.  </li>
<li>any videos tutorials on this topic(new bee on JSR233,as i m unable to find).  </li>
<li>What are the Major areas JSR223 sampler can be used.</li>
</ol>

<p>Thanks for keeping patience</p>
",https://stackoverflow.com/questions/54664464/jmeter-load-tesingmoving-from-webdriver-sampler-to-jsr223-sampler-with-groovy,"['selenium-webdriver', 'groovy', 'jmeter', 'load', 'jsr223']",351,True,0,1
"Which is better, a single &quot;insert select-join&quot; (single query) or select and then insert (two queries)","<p>I'm collecting a lot of content about users. The content has an ownerID that is different from the ids I've provided to the owner. I also have a map of ownerId to my internal ids.
I'm conflicted if I should insert the content with a single query or two queries.</p>

<p>Test schema:</p>

<pre class=""lang-sql prettyprint-override""><code>CREATE TABLE map (
   internalId INT NOT NULL,
   ownerId INT NOT NULL,
   PRIMARY KEY (internalId),
   INDEX (ownerId) 
);

Create Table content (
  internalId INT NOT NULL,
  contentId INT NOT NULL,
  PRIMARY KEY (internalId, contentId)
 );

INSERT INTO map (internalId, ownerId) VALUES (1,100), (2,1000)
</code></pre>

<p>Option 1: Select the relevant rows from the map, then build an insert query.
Option 2: The following query works (the internal select statements with the UNION ALL are generated programmatically):</p>

<pre class=""lang-sql prettyprint-override""><code>INSERT IGNORE INTO content (SELECT internalId, contentId FROM (SELECT 101 AS contentId, 100 AS ownerId UNION ALL
     SELECT 102 AS contentId, 100 AS ownerId UNION ALL
     SELECT 103 AS contentId, 100 AS ownerId UNION ALL
     SELECT 1001 AS contentId, 1000 AS ownerId) AS s_q
     INNER JOIN map USING (ownerId));
</code></pre>

<p>The second option is a bit faster (single query and all), but I haven't done any stress testing so not sure how it would handle cases where second table is locked due to inserts or updates.
The first option is easier to read and debug (and handle deadlocks - you just sort the data beforehand. I don't know if there are any guarantees in this regard about the second option).
Are there other aspects I haven't noticed?</p>
",https://stackoverflow.com/questions/58407956/which-is-better-a-single-insert-select-join-single-query-or-select-and-then,"['mysql', 'sql', 'insert']",39,False,0,1
How can the AWS Lambda concurrent execution limit be reached?,"<p><strong>UPDATE</strong></p>

<p>The original test code below is largely correct, but in NodeJS the various AWS services should be setup a bit differently as per the <a href=""https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-configuring-maxsockets.html"" rel=""nofollow noreferrer"">SDK link</a> provided by @Michael-sqlbot</p>

<pre><code>// manager
const AWS = require(""aws-sdk"")
const https = require('https');
const agent = new https.Agent({
    maxSockets: 498 // workers hit this level; expect plus 1 for the manager instance
});
const lambda = new AWS.Lambda({
    apiVersion: '2015-03-31',
    region: 'us-east-2', // Initial concurrency burst limit = 500
    httpOptions: {   // &lt;--- replace the default of 50 (https) by
        agent: agent // &lt;--- plugging the modified Agent into the service
    }
})
// NOW begin the manager handler code
</code></pre>

<hr>

<p>In planning for a new service, I am doing some preliminary stress testing. After reading about <a href=""https://docs.aws.amazon.com/lambda/latest/dg/concurrent-executions.html"" rel=""nofollow noreferrer"">the 1,000 concurrent execution limit</a> per account and the <a href=""https://docs.aws.amazon.com/lambda/latest/dg/scaling.html"" rel=""nofollow noreferrer"">initial burst rate</a> (which in us-east-2 is 500), I was expecting to achieve at least the 500 burst concurrent executions right away. The screenshot below of CloudWatch's Lambda metric shows otherwise. <em>I cannot get past 51 concurrent executions no matter what mix of parameters I try</em>. Here's the test code:</p>

<pre><code>// worker
exports.handler = async (event) =&gt; {
    // declare sleep promise
    const sleep = (ms) =&gt; new Promise((resolve) =&gt; setTimeout(resolve, ms));

    // return after one second
    let nStart = new Date().getTime()
    await sleep(1000)
    return new Date().getTime() - nStart; // report the exact ms the sleep actually took
};

// manager
exports.handler = async(event) =&gt; {
    const invokeWorker = async() =&gt; {
        try {
            let lambda = new AWS.Lambda() // NO! DO NOT DO THIS, SEE UPDATE ABOVE
            var params = {
                FunctionName: ""worker-function"",
                InvocationType: ""RequestResponse"",
                LogType: ""None""
            };
            return await lambda.invoke(params).promise()

        }
        catch (error) {
            console.log(error)
        }
    };

    try {
        let nStart = new Date().getTime()
        let aPromises = []

        // invoke workers
        for (var i = 1; i &lt;= 3000; i++) {
            aPromises.push(invokeWorker())
        }

        // record time to complete spawning
        let nSpawnMs = new Date().getTime() - nStart

        // wait for the workers to ALL return
        let aResponses = await Promise.all(aPromises)

        // sum all the actual sleep times
        const reducer = (accumulator, response) =&gt; { return accumulator + parseInt(response.Payload) };
        let nTotalWorkMs = aResponses.reduce(reducer, 0)

        // show me
        let nTotalET = new Date().getTime() - nStart
        return {
            jobsCount: aResponses.length,
            spawnCompletionMs: nSpawnMs,
            spawnCompletionPct: `${Math.floor(nSpawnMs / nTotalET * 10000) / 100}%`,
            totalElapsedMs: nTotalET,
            totalWorkMs: nTotalWorkMs,
            parallelRatio: Math.floor(nTotalET / nTotalWorkMs * 1000) / 1000
        }
    }

    catch (error) {
        console.log(error)
    }
};

Response:
{
  ""jobsCount"": 3000,
  ""spawnCompletionMs"": 1879,
  ""spawnCompletionPct"": ""2.91%"",
  ""totalElapsedMs"": 64546,
  ""totalWorkMs"": 3004205,
  ""parallelRatio"": 0.021
}

Request ID:
""43f31584-238e-4af9-9c5d-95ccab22ae84""
</code></pre>

<p>Am I hitting a different limit that I have not mentioned? Is there a flaw in my test code? I was attempting to hit the limit here with 3,000 workers, but there was NO throttling encountered, which I guess is due to the Asynchronous invocation retry behaviour.</p>

<p><strong>Edit</strong>: <em>There is no VPC involved on either Lambda; the setting in the select input is ""No VPC"".</em></p>

<p><strong>Edit</strong>: Showing Cloudwatch before and after the fix</p>

<p><a href=""https://i.stack.imgur.com/JYvvb.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/JYvvb.png"" alt=""Cannot exceed 51 Concurrent Executions""></a>
<a href=""https://i.stack.imgur.com/bRGS7.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/bRGS7.png"" alt=""With a fixed Agent, success is achieved""></a></p>
",https://stackoverflow.com/questions/54629780/how-can-the-aws-lambda-concurrent-execution-limit-be-reached,"['node.js', 'amazon-web-services', 'aws-lambda']",1862,True,5,2
Speed up Google script value access and metadata?,"<p>I have written a number of functions that look at cell values and use them to set other cell values or developer metadata, as well as look at the metadata to decide what to do. As I was stress testing the functionality of my spreadsheet I ran into problems when running a script for a big range of cells would exceed the <a href=""https://developers.google.com/apps-script/guides/services/quotas#current_limitations"" rel=""nofollow noreferrer"">maximum runtime</a>.<br>
Looking into it, I noticed that getting and setting cell values, of a predefined cell, takes about 0.2 seconds, and finding and setting metadata takes about 0.5 - 0.8 seconds.<br>
I wrote an example script to show this but for some reason the times are shorter in that, i don't know if that has to do with what is in the cell or that the spreadsheet is smaller or ....<br>
However changing a value in a sheet that already has some metadata for the specified key and triggering this script:  </p>

<pre><code>function onEdit(e){
  timetest(e)
}

function timetest(e) {
  var range = e.range;
  var sheet = range.getSheet();
  for (var r = 1; r &lt;= range.getNumRows(); r++){
    for (var c = 1; c &lt;= range.getNumColumns(); c++){
      var cell = range.getCell(r, c);
      var value = cell.getValue();
      var new_val = value + ' ' + r + c;
      cell.setValue(new_val);
    }
    var metadata = sheet.createDeveloperMetadataFinder().withKey(""Updated"").find()
    if (metadata.length &gt; 0){
      metadata[0].setValue(new_val)
    } else {
      sheet.addDeveloperMetadata(""Updated"", new_val, SpreadsheetApp.DeveloperMetadataVisibility.DOCUMENT);
    }
  }
}
</code></pre>

<p>gives this</p>

<pre><code>[19-11-21 01:05:44:105 PST] Starting execution
[19-11-21 01:05:44:252 PST] PropertiesService.getDocumentProperties() [0.14 seconds]
[19-11-21 01:05:44:253 PST] SpreadsheetApp.Range.getSheet() [0 seconds]
[19-11-21 01:05:44:254 PST] SpreadsheetApp.Range.getNumRows() [0 seconds]
[19-11-21 01:05:44:254 PST] SpreadsheetApp.Range.getNumColumns() [0 seconds]
[19-11-21 01:05:44:255 PST] SpreadsheetApp.Range.getCell([1, 1]) [0 seconds]
[19-11-21 01:05:44:357 PST] SpreadsheetApp.Range.getValue() [0.101 seconds]
[19-11-21 01:05:44:413 PST] SpreadsheetApp.Range.setValue([p 12 11]) [0.056 seconds]
[19-11-21 01:05:44:413 PST] SpreadsheetApp.Range.getNumColumns() [0 seconds]
[19-11-21 01:05:44:414 PST] SpreadsheetApp.Sheet.createDeveloperMetadataFinder() [0 seconds]
[19-11-21 01:05:44:414 PST] SpreadsheetApp.DeveloperMetadataFinder.withKey([Updated]) [0 seconds]
[19-11-21 01:05:44:682 PST] SpreadsheetApp.DeveloperMetadataFinder.find() [0.267 seconds]
[19-11-21 01:05:44:836 PST] SpreadsheetApp.DeveloperMetadata.setValue([p 12 11]) [0.153 seconds]
</code></pre>

<p>Any ideas on what makes these calls slow? Is there a way to optimize the code so that they run as fast as possible?</p>
",https://stackoverflow.com/questions/58971270/speed-up-google-script-value-access-and-metadata,"['google-apps-script', 'google-sheets']",201,False,0,1
Should you write unit tests for code that tests application behavior?,"<p>I'm wondering if you should or should not write unit tests for code that's part of the main application and can be called by the customer even though the code is only responsible for stress testing part of the application. </p>

<p>Actually, I should maybe explain why I'm asking: I used to have the opinion all production code should have unit tests no matter what the code does. But I now got two people telling me not to write unit tests for code like this and one of them is a person that teaches others about TDD and unit testing and whom I respect a lot. His explanation was:
'The purpose of unit tests is to verify good design practices, provide internal documentation of understanding of requirements and design decisions, and serve as a continuous proof of functional correctness of units of code (not necessarily the functional correctness of the entire deliverable). That had nothing to do with stress tests, that are only aimed to produce an artifact of proof of performance or reliability.'</p>

<p>So I'm now really confused if my thinking of all production code needing unit tests was wrong for years.</p>
",https://stackoverflow.com/questions/58072967/should-you-write-unit-tests-for-code-that-tests-application-behavior,['unit-testing'],25,False,-2,1
puppeteer-cluster: Setting a timeout on individual execution tasks,"<p>I'm trying to get individual tasks to throw a time-out during stress testing to see what my calling program will do. However, my cluster keeps tasks fresh indefinitely. It appears to queue all my <code>cluster.execute</code> calls which then are kept in memory and return their results to listeners that have long since disconnected.</p>

<p>The docs state:   </p>

<p><code>timeout &lt;number&gt; Specify a timeout for all tasks. Defaults to 30000 (30 seconds).</code></p>

<p>My cluster launch configuration:</p>

<pre><code>const cluster = await Cluster.launch({
    concurrency: Cluster.CONCURRENCY_CONTEXT,
    maxConcurrency: 1,
    timeout: 1000 //milliseconds
});
</code></pre>

<p>I'm calling the queuing mechanism using:</p>

<pre><code>const pdf = await cluster.execute(html, makePdf);
</code></pre>

<p>Where makePdf is an <code>async</code> function that expects a HTML string, fills a page with it and prints a PDF using the default <code>puppeteer</code>. </p>

<pre><code>const makePdf = async ({ page, data: html, worker }) =&gt; {
    await page.setContent(html);
    let pdf = await page.pdf({});
    console.log('worker ' + worker.id + ' task ' + count);
    return pdf;
};
</code></pre>

<p>I sort of expected the queue to start emptying itself until it found a task that didn't exceed its timeout value. I've tried setting the <code>timeout</code> to 1 ms but this doesn't trigger a timeout either. I've tried moving this code to a <code>cluster.task</code> as described in the examples to see if that would trigger the setting, but no such luck. How do I get already queued requests to time out? Does this even work if I'm not scraping websites or connecting to anything?</p>

<p>I'm considering to pass a timestamp along with my tasks so it can skip doing anything for requests that have expired on the calling side, but I'd rather use built-in options wherever possible.</p>

<p>EDIT:</p>

<p>Thanks to Thomas's clarification I've decided to build this little optimization to prevent tasks where the listeners are long gone from executing.</p>

<p>Swap the content of <code>data</code> from just html with a json that has both the url and timestamp:</p>

<pre><code>let timestamp = new Date();
await cluster.execute({html, timestamp});
</code></pre>

<p>Ignore any queued task where the listener has timed out:</p>

<pre><code>const makePdf = async ({ page, data: { html, timestamp }, worker }) =&gt; {
    let time_since_call = (new Date() - timestamp);
    if (time_since_call &lt; timeout_ms) {
        await page.setContent(html);
        let pdf = await page.pdf({});
        return pdf;
    } 
};
</code></pre>
",https://stackoverflow.com/questions/57394609/puppeteer-cluster-setting-a-timeout-on-individual-execution-tasks,"['javascript', 'node.js', 'puppeteer-cluster']",3420,True,1,1
Spring WebFlux with MongoDB - throttling SSE clients,"<p>I am working on a simple chat service ran by Spring Boot 2.1.1 with WebFlux, Reactor 3.2.3, Mongo 3.8.2 and Netty 4.1.31.</p>

<p>Each chat room has 2 collections - messages archive and a capped collection with current events (eg. new message event, user typing indicators etc.). The capped collection has 100 elements and I am using tail() method of ReactiveMongoTemplate to retrieve latest events.</p>

<p>The service exposes 2 kinds of endpoints for retrieving the recent events: SSE and for polling. I have done some stress testing with 2000 concurrent users which apart from listening to the chat, were spamming tons of events.</p>

<p><strong>The observations are:</strong></p>

<ul>
<li>polling every 2 seconds brings a bit of stress to the service (~40% CPU usage during the test) and almost no stress to the MongoDB (~4%)</li>
<li>listening via SSE maxes out the MongoDB (~90%), also stresses the service (which tries to use the rest of available resources), but Mongo is particularly struggling and overall the service becomes almost unresponsive.</li>
</ul>

<p>The observation seems obvious, because when I have connected via SSE during the test, it has updated me almost instantly when new event arrived - basically SSE was hundreds of times more responsive than polling every 2 seconds.</p>

<p><strong>The question is:</strong></p>

<p>Given that the client is ultimately the subscriber (or at least I think it is given by limited knowledge), can I somehow throttle the rate of publishing messages by ReactiveMongoTemplate? Or somehow decrease the demand for new events without having to do that client-side?</p>

<p>I have been trying my luck with Flux buffering and cache'ing, however it caused even more stress...</p>

<p><strong>Code:</strong></p>

<pre><code>// ChatRepository.java

private static final Query chatEventsQuery = new Query();

public Flux&lt;ChatEvent&gt; getChatEventsStream(String chatId) {
    return reactiveMongoTemplate.tail(
            chatEventsQuery,
            ChatEvent.class,
            chatId
    );
}
</code></pre>

<p>,</p>

<pre><code>// ChatHandler.java

public Mono&lt;ServerResponse&gt; getChatStream(ServerRequest request) {

    String chatId = request.pathVariable(CHAT_ID_PATH_VARIABLE);
    String username = getUsername(request);

    Flux&lt;ServerSentEvent&gt; chatEventsStream = chatRepository
            .getChatEventsStream(chatId)
            .map(addUserSpecificPropsToChatEvent(username))
            .map(event -&gt; ServerSentEvent.&lt;ChatEvent&gt;builder()
                    .event(event.getType().getEventName())
                    .data(event)
                    .build());

    log.debug(""\nExposing chat stream\nchat: {}\nuser: {}"", chatId, username);

    return ServerResponse.ok().body(
            chatEventsStream,
            ServerSentEvent.class
    );
}
</code></pre>

<p>,</p>

<pre><code>// ChatRouter.java

RouterFunction&lt;ServerResponse&gt; routes(ChatHandler handler) {
    return route(GET(""/api/chat/{chatId}/stream""), handler::getChatStream);
}
</code></pre>
",https://stackoverflow.com/questions/54018502/spring-webflux-with-mongodb-throttling-sse-clients,"['spring-webflux', 'project-reactor', 'spring-mongodb', 'reactor-netty']",1444,True,3,1
Google Cloud SQL is SLOW: mysql instance with 10GB RAM is 20x slower than Macbook Pro configured with 125MB ram,"<p>We dumped our table per Google Cloud SQL instructions and imported it into a second generation Google Cloud SQL instance. </p>

<p>We were very excited to see how our numbers would be running on ""google hardware"". </p>

<p>After stress testing our Rails app with Apache <code>ab</code> and seeing 150ms higher completed times, we noticed ActiveRecord was taking from 30ms to 50ms more than our production server (bare metal) in the same pages.</p>

<p>While we dug deeper, what really blew our minds were simple count queries like this:</p>

<pre><code>GOOGLE CLOUD SQL - db-n1-standard-4 (4vcpu and 15GB RAM)

1. Cold query

mysql&gt; SELECT COUNT(*) FROM `event_log`;
+----------+
| COUNT(*) |
+----------+
|  3998050 |
+----------+
1 row in set (19.26 sec)

2. Repeat query

mysql&gt; SELECT COUNT(*) FROM `event_log`;
+----------+
| COUNT(*) |
+----------+
|  3998050 |
+----------+
1 row in set (1.16 sec)

SELECT @@innodb_buffer_pool_size/1024/1024/1024;
+------------------------------------------+
| @@innodb_buffer_pool_size/1024/1024/1024 |
+------------------------------------------+
|                          10.500000000000 |
+------------------------------------------+
1 row in set (0.00 sec)


</code></pre>

<p>I can then repeat the query multiple times and the performance is the same. </p>

<p>Running the same query in my macbook pro 2017 with the exact same dump:</p>

<pre><code>MACBOOK PRO 2017

1. Cold query

mysql&gt; SELECT COUNT(*) FROM `event_log`;
+----------+
| COUNT(*) |
+----------+
|  3998050 |
+----------+
1 row in set (1.51 sec)

2. Repeat query

mysql&gt; SELECT COUNT(*) FROM `event_log`;
+----------+
| COUNT(*) |
+----------+
|  3998050 |
+----------+
1 row in set (0,51 sec)

SELECT @@innodb_buffer_pool_size/1024/1024/1024;
+------------------------------------------+
| @@innodb_buffer_pool_size/1024/1024/1024 |
+------------------------------------------+
|                           0.125000000000 |
+------------------------------------------+
1 row in set (0,03 sec)



</code></pre>

<p>What makes it even more absurd is that, as you can see above, I haven't tuned anything from my default mysql install, so it's using only 125MB of RAM in my Macbook, while the Google Cloud instance has 10GB of RAM available.</p>

<p>We tried increasing Google Cloud SQL instance size up to db-n1-highmen-8 (8vCPU with 52GB ram!) to no increase of performance (if we decrease from db-n1-standard-4 we do see a decrease in performance). </p>

<p>Last but not least, using <a href=""https://stackoverflow.com/questions/1733507/how-to-get-size-of-mysql-database"">this question</a> we can confirm that our database has only 46GB, but during the import the storage usage in the google cloud sql kept growing until reaching absurd 74GB... we don't know if that's because of binary logging (which is ON on google cloud SQL by default and off on my local machine). </p>

<p>So .. isn't anyone using Google Cloud sql on production? :)</p>

<p><strong>UPDATE</strong>: we used the exact same .sql dump and loaded it into a db.r4.large AWS RDS (so same cpu / ram) and got consistent 0,50s performance in the query, and it also didnt consume more then 46GB in the instance.</p>
",https://stackoverflow.com/questions/54503324/google-cloud-sql-is-slow-mysql-instance-with-10gb-ram-is-20x-slower-than-macboo,['mysql'],5370,True,8,1
Kafka: KafkaConsumer not able to pull all records,"<p>I'm pretty new at Kafka. </p>

<p>For the purpose of stress testing my cluster, and building operational experience, I created two simple Java applications: one that repeatedly publishes messages to a topic (a sequence of integers) and another application that loads the entire topic (all records) and verifies that the sequence is complete. Expectation is that no messages get lost due to operations on the cluster (restart a node, replacing a node, topics partitions reconfigurations, etc).</p>

<p>The topic ""sequence"" has two partitions, and replication factor 3. The cluster is made of 3 virtual nodes (its for testing purposes, hence they are running on the same machine). The topic is configured to retain all messages (<code>retention.ms</code> set to <code>-1</code>)</p>

<p>I currently have two issues, that I have difficulties figuring out:</p>

<ol>
<li><p>If I use <code>bin/kafka-console-consumer.sh --bootstrap-server kafka-test-server:9090,kafka-test-server:9091,kafka-test-server:9092 --topic sequence --from-beginning</code> I see ALL messages (even though not ordered, as expected) loaded on console. On the other hand, if I use the consumer application that I wrote, I see different results being loaded at each cycle: <a href=""https://i.stack.imgur.com/tMK10.png"" rel=""nofollow noreferrer"">https://i.stack.imgur.com/tMK10.png</a> - In the console output, the first line after the divisor is a call to <code>records.partitions()</code>, hence records are only sometimes pulled from both partitions. Why and why is the java app not behaving like <code>bin/kafka-console-consumer.sh</code>?</p></li>
<li><p>When the topic gets to big, the <code>bin/kafka-console-consumer.sh</code> is still able to show all messages, while the application is able to load only about 18'000 messages. I have tried playing around with 
the various consumer-side configurations, with no progress. Again, the question is why is there a difference?</p></li>
</ol>

<p>Thank you in advance for any hint!</p>

<p>Here are for ref. the two app discussed:</p>

<pre class=""lang-java prettyprint-override""><code>package ch.demo.toys;

import java.io.FileInputStream;
import java.util.Properties;
import java.util.concurrent.Future;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.producer.Callback;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.RecordMetadata;

public class SequenceProducer {
    public static void main(String[] args) throws Exception {
        Properties properties = new Properties();

        properties.load(new FileInputStream(""toy.properties""));

        properties.put(""key.serializer"", ""org.apache.kafka.common.serialization.IntegerSerializer"");
        properties.put(""value.serializer"", ""org.apache.kafka.common.serialization.StringSerializer"");
        properties.put(""acks"", ""1"");
        properties.put(""retries"", ""3"");
        properties.put(""compression.type"", ""snappy"");
        properties.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, 1);

        for (Integer sequence_i = 0; true; sequence_i++) {
            try(Producer&lt;Integer, String&gt; producer = new KafkaProducer&lt;&gt;(properties)) {

                ProducerRecord&lt;Integer, String&gt; record = new ProducerRecord&lt;&gt;(""sequence"", sequence_i, ""Sequence number: "" + String.valueOf(sequence_i));

                Future&lt;RecordMetadata&gt; sendFuture = producer.send(record, (metadata, exception) -&gt; {

                    System.out.println(""Adding "" + record.key() + "" to partition "" + metadata.partition());

                    if (exception != null) {
                        exception.printStackTrace();
                    }
                });
            }

            Thread.sleep(200);
        }
    }
}
</code></pre>

<pre class=""lang-java prettyprint-override""><code>package ch.demo.toys;

import java.io.FileInputStream;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Properties;
import java.util.stream.Collectors;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.TopicPartition;

public class CarthusianConsumer {
    private static Properties getProperties() throws Exception {
        Properties properties = new Properties();
        properties.load(new FileInputStream(""toy.properties""));

        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,    org.apache.kafka.common.serialization.IntegerDeserializer.class);
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,  org.apache.kafka.common.serialization.StringDeserializer.class);
        properties.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG,          Integer.MAX_VALUE);
        properties.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG,        60 * 1000);
        properties.put(ConsumerConfig.GROUP_ID_CONFIG,                  ""carthusian-consumer"");
        properties.put(ConsumerConfig.FETCH_MAX_WAIT_MS_CONFIG,         60 * 1000);
        properties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG,        false);
        properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,         ""earliest"");
        properties.put(ConsumerConfig.FETCH_MAX_BYTES_CONFIG,           1024 * 1024 * 1024);

        return properties;
    }

    private static boolean checkConsistency(List&lt;Integer&gt; sequence) {
        Collections.sort(sequence);

        Iterator&lt;Integer&gt; iterator = sequence.iterator();

        int control = 0;
        while(iterator.hasNext()) {
            int value = iterator.next();
            if (value != control) {
                System.out.println("""");
                System.out.println(""Gap found:"");
                System.out.println(""\tSequence: "" + value);
                System.out.println(""\tControl: "" + control);
                return false;
            }

            control++;
        }

        System.out.print(""."");
        return true;
    }

    public static void main(String[] args) throws Exception {

        // Step 1: create a base consumer object
        Consumer&lt;Integer, String&gt; consumer = new KafkaConsumer&lt;&gt;(getProperties());

        // Step 2: load topic configuration and build list of TopicPartitons
        List&lt;TopicPartition&gt; topicPartitions = consumer
                .partitionsFor(""sequence"")
                .stream()
                .parallel()
                .map(partitionInfo -&gt; new TopicPartition(partitionInfo.topic(), partitionInfo.partition()))
                .collect(Collectors.toList());


        while (true) {
            List&lt;Integer&gt; sequence = new ArrayList&lt;&gt;();

            for (TopicPartition topicPartition : topicPartitions) {

                // Step 3. specify the topic-partition to ""read"" from
                // System.out.println(""Partition specified: "" + topicPartition);
                consumer.assign(Arrays.asList(topicPartition));

                // Step 4. set offset at the beginning
                consumer.seekToBeginning(Arrays.asList(topicPartition));

                // Step 5. get all records from topic-partition
                ConsumerRecords&lt;Integer, String&gt; records = consumer.poll(Duration.ofMillis(Long.MAX_VALUE));

                // System.out.println(""\tCount: "" + records.count());
                // System.out.println(""\tPartitions: "" + records.partitions());

                records.forEach(record -&gt; { sequence.add(record.key()); });
            }


            System.out.println(sequence.size());

            checkConsistency(sequence);

            Thread.sleep(2500);
        }
    }
}
</code></pre>
",https://stackoverflow.com/questions/56899452/kafka-kafkaconsumer-not-able-to-pull-all-records,"['apache-kafka', 'kafka-consumer-api']",1009,True,0,2
Cassandra data compacted until deletion with no TTL set up?,"<p>I was testing node repair on my Cassandra cluster (v3.11.5) while simultaneously stress-testing it with cassandra-stress (v3.11.4). The disk space run out and the repair failed. As a result gossip got disabled on the nodes. Sstables that were being anticompacted got cleaned up (effectively = deleted), which dropped the disk usage by ~half (to ~1.5TB per node) within a minute. And this I understand.</p>

<p>What I do not undestand is what happened next. The sstables started getting continuously compacted into smaller ones and eventually deleted. As a result the disk usage continued to drop (this time slowly), after a day or so it went from ~1.5TB per node to ~50GB per node. The data that was residing in the cluster was randomly generated by the cassandra-stress, so I see no way to confirm whether it's intact, however I find highly unlikely that it is, as the disk usage dropped that much. Also I have no TTL set up (at least that I would know of, might be missing something), so I would not expect the data being deleted. But I believe this is the case.</p>

<p>Anyway, can anyone point me to what is happening?</p>

<p>Table schema:</p>

<pre><code>&gt; desc test-table1;

CREATE TABLE test-keyspace1.test-table1 (
    event_uuid uuid,
    create_date timestamp,
    action text,
    business_profile_id int,
    client_uuid uuid,
    label text,
    params text,
    unique_id int,
    PRIMARY KEY (event_uuid, create_date)
) WITH CLUSTERING ORDER BY (create_date DESC)
    AND bloom_filter_fp_chance = 0.1
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.DeflateCompressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';
</code></pre>

<p>Logs:</p>

<pre><code>DEBUG [CompactionExecutor:7] 2019-11-23 20:17:19,828 CompactionTask.java:255 - Compacted (59ddec80-0e20-11ea-9612-67e94033cb24) 4 sstables to [/data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3259-big,] to level=0.  93.264GiB to 25.190GiB (~27% of original) in 5,970,059ms.  Read Throughput = 15.997MiB/s, Write Throughput = 4.321MiB/s, Row Throughput = ~909/s.  1,256,595 total partitions merged to 339,390.  Partition merge counts were {2:27340, 3:46285, 4:265765, }
(...)
DEBUG [CompactionExecutor:7] 2019-11-24 03:50:14,820 CompactionTask.java:255 - Compacted (e1bd7f50-0e4b-11ea-9612-67e94033cb24) 32 sstables to [/data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3301-big,] to level=0.  114.787GiB to 25.150GiB (~21% of original) in 14,448,734ms.  Read Throughput = 8.135MiB/s, Write Throughput = 1.782MiB/s, Row Throughput = ~375/s.  1,546,722 total partitions merged to 338,859.  Partition merge counts were {1:12732, 2:42441, 3:78598, 4:50454, 5:36032, 6:52989, 7:21216, 8:34681, 9:9716, }
DEBUG [CompactionExecutor:15] 2019-11-24 03:50:14,852 LeveledManifest.java:423 - L0 is too far behind, performing size-tiering there first
DEBUG [CompactionExecutor:15] 2019-11-24 03:50:14,852 CompactionTask.java:155 - Compacting (85e06040-0e6d-11ea-9612-67e94033cb24) [/data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3259-big-Data.db:level=0, /data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3299-big-Data.db:level=0, /data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3298-big-Data.db:level=0, /data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3300-big-Data.db:level=0, /data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3301-big-Data.db:level=0,]
(...)
DEBUG [NonPeriodicTasks:1] 2019-11-24 06:02:50,117 SSTable.java:105 - Deleting sstable: /data/cassandra/data/test-keyspace1/test-table1-f592e9600b9511eab562b36ee84fdea9/md-3259-big
</code></pre>

<p><strong>edit:</strong></p>

<p>I performed some additional testing. To my best knowledge there is no TTL set up, see query result straight after cassandra-stress started inserting data:</p>

<pre><code>&gt; SELECT event_uuid, create_date, ttl(action), ttl(business_profile_id), ttl(client_uuid), ttl(label), ttl(params), ttl(unique_id) FROM test-table1 LIMIT 1;

 event_uuid                           | create_date                     | ttl(action) | ttl(business_profile_id) | ttl(client_uuid) | ttl(label) | ttl(params) | ttl(unique_id)
--------------------------------------+---------------------------------+-------------+--------------------------+------------------+------------+-------------+----------------
 00000000-001b-adf7-0000-0000001badf7 | 2018-01-10 10:08:45.476000+0000 |        null |                     null |             null |       null |        null |           null
</code></pre>

<p>So neither TTL nor tombstones deletion should be related to the issue. It's likely that there are no duplicates, as the data is highly randomized. No Replication Factor changes were made, as well.</p>

<p>What I found out is that the data volume decrease starts every time after cassandra-stress gets stopped. Sadly, still don't know the exact reason.</p>
",https://stackoverflow.com/questions/59034199/cassandra-data-compacted-until-deletion-with-no-ttl-set-up,['cassandra'],310,False,0,1
C# events getting lost,"<p>I've been stress testing my C# Windows Forms and WPF programs together with ""HeavyLoad"". HeavyLoad is an application to stress test your PC in order to check if things keep running reliably. Which isn't the case with my program.</p>

<p>What I'm seeing is that whenever I run HeavyLoad, using all available cores, my program starts to lose events. This also happens with the following simple code:</p>

<p><strong>XAML:</strong></p>

<pre><code>&lt;Window x:Class=""MultithreadedWPF.MainWindow""
        xmlns=""http://schemas.microsoft.com/winfx/2006/xaml/presentation""
        xmlns:x=""http://schemas.microsoft.com/winfx/2006/xaml""
        xmlns:d=""http://schemas.microsoft.com/expression/blend/2008""
        xmlns:mc=""http://schemas.openxmlformats.org/markup-compatibility/2006""
        xmlns:local=""clr-namespace:MultithreadedWPF""
        mc:Ignorable=""d""
        Title=""MainWindow"" Height=""450"" Width=""800""&gt;
    &lt;StackPanel&gt;
        &lt;TextBox Text=""{Binding MyText, UpdateSourceTrigger=PropertyChanged}""&gt;&lt;/TextBox&gt;
    &lt;/StackPanel&gt;
&lt;/Window&gt;
</code></pre>

<p><strong>Code behind:</strong></p>

<pre><code>/// &lt;summary&gt;
/// Interaction logic for MainWindow.xaml
/// &lt;/summary&gt;
public partial class MainWindow : Window, INotifyPropertyChanged
{
    public event PropertyChangedEventHandler PropertyChanged;

    private string _myText = string.Empty;
    /// &lt;summary&gt;
    /// This property is bound to a text box in my user interface.
    /// &lt;/summary&gt;
    public string MyText
    {
        get { return _myText; }
        set
        {
            _myText = value;

            if (PropertyChanged != null)
                PropertyChanged(this, new PropertyChangedEventArgs(""MyText""));
        }
    }

    public MainWindow()
    {
        InitializeComponent();

        //Create a timer to fire every second.
        Timer timer = new Timer(1000);
        timer.AutoReset = true;
        timer.Elapsed += Timer_Elapsed;
        timer.Start();

        DataContext = this;
    }

    /// &lt;summary&gt;
    /// The timer's previously elapsed time.
    /// &lt;/summary&gt;
    DateTime previous = DateTime.Now;

    private void Timer_Elapsed(object sender, ElapsedEventArgs e)
    {
        //Save the current date and time.
        DateTime now = DateTime.Now;

        //Set the text accordingly.
        MyText = $""{now.ToString(""HH:mm:ss"")}, delta: {now - previous}"";

        //Save the current time to calculate the next delta.
        previous = now;
    }
}
</code></pre>

<p>This happens in Windows Forms as well as WPF.</p>

<hr>

<h2><strong>What I am seeing:</strong></h2>

<p><strong>- When running without HeavyLoad:</strong></p>

<ol>
<li>If my program is focused: the delta is around 1 second.</li>
<li>If my program isn't focused: the delta is around 1 second.</li>
</ol>

<p><strong>- When running HeavyLoad:</strong></p>

<ol>
<li>If my program is focused: the delta is somewhere between 1-3 seconds. </li>
<li>If my program isn't focused: the delta is somewhere between 3-15 seconds.</li>
</ol>

<p>This means not only is the user interface not updating, the event in the code behind isn't running or getting lost. Otherwise the delta would always be around 1 second.</p>

<hr>

<h2>The question:</h2>

<blockquote>
  <p>EDIT: I was using the System.Timers.Timer in my example. Not a
  DispatcherTimer. After switching to a timeapi.h timer (AccurateTimer),
  I notice that events are delayed instead of missed.</p>
  
  <p>I am using timers as an example. In my other programs the same issues
  occur with events received from TCP clients. In the real-life
  scenario's I am getting the same issues without running HeavyLoad,
  perhaps through Windows Update or other processes. We try to minimize
  the other processes as much as possible. There are no other programs
  (except for Windows services and such) running during the execution of
  our program.</p>
</blockquote>

<p>Is there a way around this issue? Perhaps by forcing our program to not run on a certain core? Using a different Windows version (e.g. LTSC, IoT or whatever)?</p>

<p>Putting my program in a higher (or highest) priority does not seem to have enough of an effect.</p>
",https://stackoverflow.com/questions/58397206/c-events-getting-lost,"['c#', 'wpf', 'windows', 'winforms', 'events']",467,False,0,0
Increasing limit of outgoing TCP connections on MacOS,"<p>Short version of question is: How to tune\configure macOS (Mojave 10.14.3) settings to allow more then 10k outgoing TCP connections per process and more then 16k connections in total.</p>

<p>Details:<br>
I'm trying to make MacBookPro (16Gb RAM, Core i7) usable for stress-testing tcp server. Server itself hosted on separate pc, so right now the question is about outgoing connections only.</p>

<p>Below advices already processed and helped me significantly increase initial os limits.<br>
1) I used [launchctl] (<a href=""https://stackoverflow.com/questions/33836092/too-many-open-files-when-executing-gatling-on-mac"">&quot;Too many open files&quot; when executing gatling on Mac</a>) to increase maxfiles limit to 1 million.<br>
2) I used <a href=""https://stackoverflow.com/questions/7578594/how-to-increase-limits-on-sockets-on-osx-for-load-testing/7580233"">sysctl</a> to set\check kern.maxfiles limits. Actually (as I understand) this is the same as #1.<br>
3) I played with <a href=""https://stackoverflow.com/questions/13988780/too-many-open-files-ulimit-already-changed"">ulimit</a>. Actually I didn't notice any effect of this tool on my OS. But any way...</p>

<p>So now I MacOS can establish ~10k connections per process and 16k total connections in the system.</p>

<p>For simplicity my tool just open TCP connections in a infinite loop and waits. </p>

<pre><code>try
{
  while (true)
  {
    CreateAndConnectSocket(); //add socket to list
    ++connectedSockets;
  }
}
catch(Exception e)
{
  LogWrite(""Connected sockets:"" + connectedSockets);
  LogWrite(e);
  WaitForAnyKey();
}
</code></pre>

<p>Then I follow below steps.</p>

<p>1) Launch server on separate PC.
2) Open two terminals on mac.<br>
3) Execute in first terminal window:    </p>

<pre><code>$ sudo launchctl limit maxfiles 1048576 1048600   
$ ulimit -S -n 1048576   
</code></pre>

<p>4) Verify that changes applied in first terminal:  </p>

<pre><code>$ ulimit -S -n  
1048576  
$ launchctl limit maxfiles  
    maxfiles    1048576        1048600      
$ sysctl kern.maxfilesperproc  
kern.maxfilesperproc: 1048576
$ sysctl kern.maxfiles  
kern.maxfiles: 1048600  
</code></pre>

<p>5) Launch ""ulimit -S -n 1048576"" in second terminal (Not sure that ulimit is required at all.)<br>
6) Verify that all changes applied in second terminal window (same as #4).
7) Launch ""test client"" in first terminal.<br>
8) Launch ""test client"" in second terminal.</p>

<p>Result:<br>
After step 7 in first terminal I can see, that tool opened 10k connections (10202 to be precise) and fell down with exception <em>""Too many open files in system""</em>.  Have no idea why opened files is an issue with 1 million limit.<br>
After step 8 in second terminal I can see that tool opened 6k connectoins and fell down with exception <em>""Can't assign requested address""</em>.   </p>

<p>While sockets remain opened (tools wait for key press), no other connections can be created in the system - browsers can't establish connections to google.com, etc.</p>

<p>And ofcourse tcp server remains accessible from another PCs.</p>

<p>Since I was able to tune ""Windows 10 Home"" for higher connection numbers, I believe that MacOS can be tuned too.</p>
",https://stackoverflow.com/questions/55366733/increasing-limit-of-outgoing-tcp-connections-on-macos,"['macos', 'sockets', 'tcp', 'configuration', 'limit']",4036,True,6,1
"C Programming :: accept() works and then doesn&#39;t, in stress-testing","<p>I am coding what I thought would be a simple C socket program.  The code will act as a primitive server.  It needs to create a listening socket, and then each time it is contacted, it needs to <strong>accept()</strong> the incoming connection, then spin off a thread to handle some basic processing.  The threading and processing part I more-or-less have working, but the <strong>accept()</strong> part breaks down after the code successfully handles some initial connections..  This is a problem, because I anticipate that this program will need to run continuously and may have to process a lot of incoming requests.</p>

<p>Some environmental stuff first:  I’m working on a Ubuntu box, and my code is compiled with GCC:</p>

<pre><code>root@ubuntu:/home/me/socketProject# gcc -v
...
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
root@ubuntu:/home/me/socketProject#
</code></pre>

<p>My <strong>main()</strong> function sets up a listening socket, then starts an infinite loop.  For every connection that comes in, it tries to <strong>accept()</strong> and then pass the work off to a detached thread:</p>

<pre><code>int main(int argc, char ** argv)
{
   int             sock = -1;
   struct          sockaddr_in address;
   int             port = 12345;
   connection_t*   connection;
   pthread_t       thread;

   // Create the listening socket
   sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
   if (sock &lt;= 0){
      fprintf(stderr, ""%s: error: cannot create socket\n"", argv[0]);
      return -3;
   }

   // Bind socket to port
   address.sin_family = AF_INET;
   address.sin_addr.s_addr = INADDR_ANY;
   address.sin_port = htons(port);
   if (bind(sock, (struct sockaddr *)&amp;address, sizeof(struct sockaddr_in)) &lt; 0){
      fprintf(stderr, ""%s: error: cannot bind socket to port %d\n"", argv[0], port);
      return -4;
   }

   // Listen on the port
   if (listen(sock, 5) &lt; 0){
      fprintf(stderr, ""%s: error: cannot listen on port\n"", argv[0]);
      return -5;
   }

   // Listen forever...
   while (1){
      // Accept incoming connections
      connection = (connection_t *)malloc(sizeof(connection_t));

      printf(""1)  [[ sock: %d -- address: %s -- addr_len: %d ]]\n"", connection-&gt;sock, &amp;connection-&gt;address.sa_data, connection-&gt;addr_len );

      connection-&gt;sock = accept(sock, &amp;connection-&gt;address, &amp;connection-&gt;addr_len);

      printf(""2)  [[ sock: %d -- address: %s -- addr_len: %d ]]\n"", connection-&gt;sock, &amp;connection-&gt;address.sa_data, connection-&gt;addr_len );

      if (connection-&gt;sock &lt;= 0){
         printf("" *** PROBLEM!!!  Value of errno: %d\n "", errno);
         free(connection);
      }
      else{
         // Start a new thread but do not wait for it
         pthread_create(&amp;thread, 0, process, (void *)connection);  // process() omitted
         pthread_detach(thread);
      }
   }
   return 0;
}
</code></pre>

<p>Pretty textbook stuff.  I worked out all the logic and got everything working perfectly when there is only one incoming connection.</p>

<p>But then I started stress-testing, and immediately realized I had a problem.  I wrote a client program that send X number of test requests, all in a row.  The above server code works fine for a few initial requests, but then breaks down.  Here’s the output:</p>

<pre><code>root@ubuntu:/home/me/socketProject# ./server
        *** 1)  [[ sock: 18954128 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 7 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18970816 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 8 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18962080 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 9 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18962112 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 10 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18954064 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 11 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18954160 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 12 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18907120 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 13 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18950880 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 14 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 15 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18957136 -- address:  -- addr_len: 0 ]]
        *** 2)  [[ sock: 17 -- address:  -- addr_len: 16 ]]
        *** 1)  [[ sock: 18957168 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
        *** 2)  [[ sock: -1 -- address:  -- addr_len: -518793662 ]]
 *** PROBLEM!!!  Value of errno: 22
        *** 1)  [[ sock: 18961856 -- address:  -- addr_len: -518793662 ]]
</code></pre>

<p>You see the problem.  In the above example, something happens after about ten successful requests, then <strong>accept()</strong> starts complaining of <strong>ERRNO 22</strong>.  A quick Google search revealed that <strong>ERRNO 22</strong> is “<strong><em>EINVAL: Invalid argument</em></strong>”, which obviously isn’t good.</p>

<p>I’ve done some simple socket programming from my student days, but I’ve never had a problem with <strong>accept()</strong> before.  I’m really puzzled.  It looks to me like N-1 requests arrive and are successfully processed by <strong>accept()</strong>.  But then the Nth appears, <strong>accept()</strong> doesn’t like it, and the socket gets corrupted.  How or why, I don’t know.</p>

<p><strong>accept()</strong> is complaining about ""invalid argument(s)"".  But the client is literally sending the exact same test message over and over.  How can the N-1 connections present valid arguments, but the Nth, N+1th, N+2th... argument be invalid?  That blows my mind.</p>

<p>Some observations:</p>

<ul>
<li><p>This is a black-and-white phenomenon.  By that, I mean that the
socket works perfectly 100% of the time, but then something bad
happens with <strong>accept()</strong>, and then the socket never works again.</p></li>
<li><p>I’ve run a lot of tests, and there’s no telling how many successful
connections I’ll get when I start a new test.  I’ve had as many as 88
successful connections before the socket stops working.  In another
test, I only had 2 successful connections before the socket went
belly-up.  The average number of successful connections before the
socket breaks is about 17.</p></li>
<li><p>My client test program does not try to space out the requests it
sends; it simply opens a socket and starts bombarding my server
program at top speed.  I’m doing this because I want to stress-test. 
But perhaps my client program is sending test requests too fast for
the server to keep up…?</p></li>
<li><p>It is also possible, I suppose, that the client program is causing
the problem.  I don’t think so, because it just sends the same test
message over and over and it never throws an error.</p></li>
</ul>

<p>Has anyone ever seen an issue like this?  I’ve been looking on SO for about an hour, but I can’t find another post where <strong>accept()</strong> works, and then doesn’t.</p>

<p>Any help/advice is appreciated.</p>
",https://stackoverflow.com/questions/57981377/c-programming-accept-works-and-then-doesnt-in-stress-testing,"['c', 'sockets', 'stress-testing']",140,True,0,1
Unable to Read/Write using pyserial on Linux (Xubuntu 18.04),"<p>I have a proprietary device that is connected to my computer with a USB cable.</p>

<p>I have been unable to successfully read (or write, or both?) when running the code on my Xubuntu installation.</p>

<p><strong>Note:</strong> The same logic works pretty much flawlessly on both <strong>Windows</strong> (tested on 7, 8 and 10) and <strong>Mac</strong> (tested on Yosemite and Mojave).</p>

<p>Here is the code I'm trying to run (hardcoded some values for simplicity):</p>

<pre><code>ser = serial.Serial('/dev/ttyACM0', 500000, timeout = 0.03, write_timeout = 0.015)
ser.write(bytes([248, 40, 0]))
ser.flush()
reply = ser.read(40)
</code></pre>

<p><strong>Current behaviour</strong></p>

<p>The device is running some code that responds to the packets that it receives. In this initial packet, my Python code is requesting a full copy of the device's state, which it receives in a whopping 40 bytes array.</p>

<p>On Linux, the following is returned instead of the requested 40 bytes:
<code>[248, 13, 10, 40, 13, 10, 0, 13, 10]</code></p>

<p>What I find interesting is that it returns the exact same bytes that I tried writing, with 13 and 10 in between them.</p>

<p><strong>Expected behaviour</strong> 
On both Mac and Windows, something like this is returned (contents are not that important):</p>

<p><code>[71, 89, 65, 32, 1, 4, 50, 0, 27, 0, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 238, 1, 255, 255, 136, 1, 255, 255, 255, 255]</code></p>

<p>The expected behaviour is the same as the behaviour on Mac and Windows.</p>

<p><strong>Extra info</strong></p>

<ul>
<li><p>Running on Python 3.5.</p></li>
<li><p>I'm using the <strong>exact same</strong> USB device <strong>and</strong> cable.</p></li>
<li><p>I've tried all ports on the computer.</p></li>
<li><p>The code is exactly the same on all operating systems, with only the port name being different due to the nature of said OSs.</p></li>
<li><p>The serial port is confirmed to be device I need (I print the name) and is confirmed to be <strong>open</strong> (I print out the serial object).</p></li>
<li><p>I've played around with <strong>timeouts</strong> and <strong>baudrates</strong>, but to no avail.</p></li>
<li><p>I've tried exchanging the read with a <code>if(ser.in_waiting): reply = ser.read()</code> but <strong>nothing really changes</strong>.</p></li>
<li><p>I have verified that the device is on port <code>/dev/ttyACM0</code>.</p></li>
<li><p>I have added permissions to that port after encountering a <strong>OSError: [Errno 13] Permission denied: '/dev/ttyACM0'</strong> error.</p></li>
<li><p>Permissions were added by using both <code>sudo chmod 666 /dev/ttyACM0</code> and <code>sudo usermod -aG dialout $USER</code> after reading some solutions on here.</p></li>
<li><p>I have experimented with <code>time.sleep()</code>.</p></li>
<li><p><strong>[EDIT] Tried out the suggestions given by <a href=""https://stackoverflow.com/a/55064963/10691720"">@michael-o</a>.</strong> No luck.</p></li>
<li><p>Some other SO questions I've researched and tried, in no particular order:</p>

<ul>
<li><p><a href=""https://stackoverflow.com/questions/49957705/serial-ports-linux-vs-windows"">Serial Ports Linux vs Windows</a></p></li>
<li><p><a href=""https://stackoverflow.com/questions/27858041/oserror-errno-13-permission-denied-dev-ttyacm0-using-pyserial-from-pyth"">OSError: [Errno 13] Permission denied: &#39;/dev/ttyACM0&#39; - using pyserial from Python to Arduino</a></p></li>
<li><p><a href=""https://stackoverflow.com/questions/49408849/pyserial-can-write-but-not-read"">pySerial can write, but not read</a></p></li>
<li><p><a href=""https://stackoverflow.com/questions/17553543/pyserial-non-blocking-read-loop"">PySerial non-blocking read loop</a></p></li>
</ul></li>
</ul>

<p>Any info/ideas will be greatly appreciated!</p>

<p><strong>[EDIT2]</strong> The problem has become even weirder. Here's what I tried today:</p>

<ul>
<li><p>First, I programmed an Arduino on a Windows machine to constantly print a number and to blink the LED if it receives anything. Then I tested on the Linux machine and everything worked great.</p></li>
<li><p>Then, we programmed our hardware to do the same - constantly print a number and blink the LED if it receives anything.</p></li>
<li><p>Even though the logic is the same, it still fails to read or write anything to it. I even tried using GtkTerm to just inspect what is being communicated between the device and the computer. Nothing.</p></li>
<li><p>However! Sending ""X"" to the device triggers a small custom-made terminal that has some built-in commands, like ""S"" for stress testing and such.</p></li>
<li><p>The important command here is ""E"", which exits the terminal. If this is done (in GtkTerm), the device suddenly starts printing that number mentioned earlier. The intuition here was that on Linux (or at least in GtkTerm), ""X"" is sent to the device for whatever reason. So we tried exiting that custom terminal.</p></li>
<li><p>This unfortunately didn't in Python. It still continues receiving the same packet as described before.</p></li>
<li><p>So... we are stuck. But it seems like it is a firmware issue <strong>bundled</strong> with some weird quirks of the operating system. Will update if we ever figure it out.</p></li>
</ul>
",https://stackoverflow.com/questions/55064724/unable-to-read-write-using-pyserial-on-linux-xubuntu-18-04,"['linux', 'python-3.x', 'ubuntu', 'pyserial']",836,True,0,2
