title,body,link,tags,view_count,is_answered,score,answer_count
Stress testing Apache Tomcat app,"<p>How can I stress test apache tomcat app? I tried using Jmeter but i get an error after a certain number of users.</p>

<p><code>org.apache.http.TruncatedChunkException: Truncated chunk ( expected size: 8094; actual size: 6479)
    at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:198)
    at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
    at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:148)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.readResponse(HTTPSamplerBase.java:1814)
    at org.apache.jmeter.protocol.http.sampler.HTTPAbstractImpl.readResponse(HTTPAbstractImpl.java:440)
    at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:433)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.followRedirects(HTTPSamplerBase.java:1542)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.resultProcessing(HTTPSamplerBase.java:1636)
    at org.apache.jmeter.protocol.http.sampler.HTTPAbstractImpl.resultProcessing(HTTPAbstractImpl.java:519)
    at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:493)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1189)
    at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1178)
    at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:491)
    at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:425)
    at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:254)
    at java.lang.Thread.run(Unknown Source)</code></p>
",https://stackoverflow.com/questions/48519253/stress-testing-apache-tomcat-app,"['tomcat', 'jmeter', 'stress-testing']",889,FALSE,0,1
C++ Sockets and stress testing,"<p>I've written a very simple socket server in C++ (MinGW) using these common functions like </p>

<pre><code>socket( PF_INET, SOCK_STREAM, 0 )...
setsockopt( s, SOL_SOCKET, SO_REUSEADDR, &amp;OptVal, sizeof( OptVal ) )...
bind( s, ( struct sockaddr * ) &amp;ServerAddress, sizeof( ServerAddress ) )...
listen( s, 10 )...
</code></pre>

<p>The handling of multiple client connections is done by</p>

<pre><code>select( s, &amp;FileDescriptorClient, NULL, NULL, &amp;tv )...
accept( Server-&gt;GetSocketHandle(), (struct sockaddr*) &amp;ClientAddress, &amp;Length )...
</code></pre>

<p>Everything looked very good and pretty,... until I decided to <strong>stress test</strong> my server.</p>

<p>My first test was a very simple client which did only one thing: connect and disconnect in an endless loop - as fast as possible. Although this test was extremly simple it failed immediately.</p>

<p>It wasn't a too big surprise to me that the server will choke on so many toggeling connections, so I added a Sleep(5) (milliseconds) in the client before each connect and disconnect and everything was OK. For the moment.</p>

<p>My questions are: </p>

<ul>
<li>How do I handle these reconnects correct?  </li>
<li>And what is the proper way
to stress test a socket server application?</li>
</ul>

<p>Right now the procedure is as follows:</p>

<ol>
<li>client: connects to server using connect(...)</li>
<li>server: the new connection is recognized by select(...) and accept(...). Every new connection is stored to a std::vector.</li>
<li>client: disconnects from server using closesocket(...) (MinGW...)</li>
<li>server: recv(...) reads 0 bytes what means the client has disconnected from server</li>
<li>server: performs a closesocket(...) and removes connection from std::vector</li>
<li>goto 1</li>
</ol>

<p>As already mentioned: this will only work when I throttle the client with sleeps. As soon as I reduce the sleeping times, the server starts skipping pt. 4 (disconnects) and stockpiles open connections down the line.</p>

<p>What is the point I am missing?</p>

<p>Edit:</p>

<ul>
<li>server runs in a single thread</li>
<li>client socket is non-blocking</li>
</ul>

<p>Edit 2:</p>

<p>As requested: the source code (cut to a minimal version):</p>

<p>Server's main.cpp:</p>

<pre><code>#include &lt;iostream&gt;
#include ""Server.h""
using namespace std;

int main()
{
    try
    {
        ServerSocket srv;

        while ( true )
        {
            srv.Open();
            srv.Run();
        }
    }
    catch( ServerException const &amp;e )
    {
        std::cout &lt;&lt; ""failed to run the server"" &lt;&lt; e.what() &lt;&lt; std::endl;
    }

    return 0;
}
</code></pre>

<p>Server.h:</p>

<pre><code>#ifndef SERVER_H
#define SERVER_H

#define SOCKETBUFLEN 10

#include &lt;string&gt;
#include &lt;cstdio&gt;
#include &lt;vector&gt;
#include &lt;memory&gt;
#include &lt;unistd.h&gt;

#define WIN32_LEAN_AND_MEAN
#include &lt;windows.h&gt;
#include &lt;winsock2.h&gt;
#include &lt;ws2tcpip.h&gt;

#include &lt;sys/types.h&gt;

class Exception : public std::exception
{
    private:
        std::string Message;
    public:
        template&lt; typename ... Arguments &gt;
        Exception( const char * AFormat, Arguments&amp;&amp; ... AArguments )
        {
            char Buffer[ 1024 ];
            sprintf( Buffer, AFormat, std::forward&lt;Arguments&gt;( AArguments ) ... );
            Message = std::string( Buffer );
        };
        const char* what() const noexcept{ return Message.c_str(); }
};

class ServerException : public Exception
{
    public:
        template&lt; typename ... Arguments &gt;
        ServerException( const char * AFormat, Arguments&amp;&amp; ... AArguments ) : Exception( AFormat, std::forward&lt; Arguments &gt;( AArguments ) ... ){}
};

class ClientException : public Exception
{
    public:
        template&lt; typename ... Arguments &gt;
        ClientException( const char * AFormat, Arguments&amp;&amp; ... AArguments ) : Exception( AFormat, std::forward&lt; Arguments &gt;( AArguments ) ... ){}
};

class Client
{
    public:
        Client();
        virtual ~Client();

        virtual bool Authenticate( const std::string &amp; AClientId ) = 0;
        virtual bool Process( const std::string &amp; AMessage ) = 0;
    protected:
        std::vector&lt; std::string &gt; Clients;
    private:
};

class ServerSocket;
class ClientSocket;
class Socket;

class ClientSockets
{
    private:
        ServerSocket *OwnerServer;
        fd_set FileDescriptorClient;
        std::vector&lt; ClientSocket * &gt; Items;
        std::vector&lt; Socket * &gt; SocketsRead;
        void Append( ClientSocket * );
        void OnClientConnect( SOCKET ASocketHandle, struct sockaddr_in AClientAddress );
    public:
        ClientSockets( ServerSocket * );
        int Count( void );
        void Remove( SOCKET ASocketHandle );
        bool HandleConnnections( void );
        void Read( void );
};

class Socket
{
    protected:
        SOCKET SocketHandle;
    public:
        Socket( SOCKET ASocketHandle = 0 ) { SocketHandle = ASocketHandle; };
        SOCKET GetSocketHandle( void ) { return SocketHandle; };
        virtual void Close( void ) = 0;
};

class ServerSocket : public Socket
{
    public:
        ServerSocket( const std::string &amp;AHost = ""127.0.0.1"", const int &amp;APort = 24442 );
        virtual ~ServerSocket();

        void Open( void );
        void Close( void ) {};
        void Run( void );

        fd_set FileDescriptorServer;
    private:
        std::string Host;
        int Port;
        ClientSockets *clientSockets;
};

class ClientSocket : public Socket
{
    private:
        ClientSockets *FClientSockets;
    public:
        ClientSocket( ClientSockets *, SOCKET ASocketHandle );
        virtual ~ClientSocket() {};

        void Close( void );
};

#endif // SERVER_H
</code></pre>

<p>Server.cpp:</p>

<pre><code>#include ""Server.h""

#include &lt;cstring&gt;
#include &lt;cstdarg&gt;
#include &lt;cstdio&gt;
#include &lt;algorithm&gt;
#include &lt;iostream&gt;
#include &lt;thread&gt;
#include &lt;chrono&gt;

#define LOG_CONNECTION_AMOUNT 100

ServerSocket::ServerSocket( const std::string &amp;AHost, const int &amp;APort ) :
    Host( AHost ),
    Port( APort ),
    clientSockets( new ClientSockets( this ) ) {}

ServerSocket::~ServerSocket()
{
    WSACleanup();
}

void ServerSocket::Open( void )
{
    WSADATA wsa;

    if ( WSAStartup( MAKEWORD( 2, 2 ), &amp;wsa ) != 0 )
    {
        throw ServerException( ""WSAStartup failed"" );
    }

    if ( ( SocketHandle = socket( PF_INET, SOCK_STREAM, 0 ) ) == INVALID_SOCKET )
    {
        throw ServerException( ""%s: unable to get a socket handle. errno: %ld"", __PRETTY_FUNCTION__, errno );
    }

    char OptVal = 1;

    if ( setsockopt( SocketHandle, SOL_SOCKET, SO_REUSEADDR, &amp;OptVal, sizeof( OptVal ) ) == -1 )
    {
        throw ServerException( ""%s: setsockopt failed"", __PRETTY_FUNCTION__ );
    }

    struct sockaddr_in ServerAddress;

    memset( &amp; ServerAddress, 0x0, sizeof( ServerAddress ) );

    ServerAddress.sin_family = AF_INET;
    ServerAddress.sin_addr.s_addr = inet_addr( Host.c_str() );
    ServerAddress.sin_port = htons( Port );

    if ( bind( SocketHandle, ( struct sockaddr * ) &amp;ServerAddress, sizeof( ServerAddress ) ) == -1 )
    {
        throw ServerException( ""%s: binding server address failed"", __PRETTY_FUNCTION__ );
    }

    if ( listen( SocketHandle, 10 ) == -1 )
    {
        throw ServerException( ""%s: listen socket failed"", __PRETTY_FUNCTION__ );
    }

    FD_ZERO( &amp;FileDescriptorServer );
    FD_SET( SocketHandle, &amp;FileDescriptorServer );

    std::cout &lt;&lt; ""socket server fd '"" &lt;&lt; SocketHandle &lt;&lt; ""' listening on "" &lt;&lt; Host &lt;&lt; "":"" &lt;&lt; Port &lt;&lt; std::endl;
}

void ServerSocket::Run( void )
{
    std::cout &lt;&lt; __PRETTY_FUNCTION__  &lt;&lt; std::endl;

    do
    {
        try
        {
            if ( !clientSockets-&gt;HandleConnnections() )
            {
                continue;
            }

            clientSockets-&gt;Read();
        }
        catch( ServerException const &amp;e )
        {
            int ErrorCode = WSAGetLastError();

            std::cout &lt;&lt; ""error while server running: "" &lt;&lt; e.what() &lt;&lt; "", code: "" &lt;&lt; ErrorCode &lt;&lt; std::endl;

            if ( ErrorCode == 10038 ) // no socket error
            {
                // continue; // not sure if this is the correct way to do...
            }

            // if ( ErrorCode == 10014 ) {} // bad address error
            // if ( ErrorCode == 10054 ) {} // connection reset by peer

            break;
        }
        catch( ... )
        {
            std::cout &lt;&lt; ""some exception occurred"" &lt;&lt; std::endl;
        }
    } while ( true );
}

ClientSockets::ClientSockets( ServerSocket * AOwnerServer )
{
    OwnerServer = AOwnerServer;
}

void ClientSockets::Append( ClientSocket *AClientSocket )
{
    Items.push_back( AClientSocket );
}

int ClientSockets::Count( void )
{
    return Items.size();
}

bool ClientSockets::HandleConnnections( void )
{
    FD_ZERO( &amp; FileDescriptorClient );

    SocketsRead.clear();
    SocketsRead.push_back( OwnerServer );

    for ( auto i : Items )
    {
        SocketsRead.push_back( i );
    }

    for ( auto Socket : SocketsRead )
    {
        if ( FD_ISSET( Socket-&gt;GetSocketHandle(), &amp; OwnerServer-&gt;FileDescriptorServer ) )
        {
            FD_SET( Socket-&gt;GetSocketHandle(), &amp; FileDescriptorClient );
        }
    }

    struct timeval tv;
    tv.tv_sec = 10;
    tv.tv_usec = 3;

    int SocketChangedCount = 0;

    if ( ( SocketChangedCount = select( 0, &amp;FileDescriptorClient, NULL, NULL, &amp;tv ) ) == -1 )
    {
        throw ServerException( ""%s: select failed"", __PRETTY_FUNCTION__ );
    }

    static int scc = 0;
    if ( scc != SocketChangedCount )
    {
        scc = SocketChangedCount;
        std::cout &lt;&lt; ""socket changes: "" &lt;&lt; SocketChangedCount &lt;&lt; std::endl;
    }


    if ( FD_ISSET( OwnerServer-&gt;GetSocketHandle(), &amp;FileDescriptorClient ) )
    {
        struct sockaddr_in ClientAddress;

        socklen_t Length = sizeof( ClientAddress );

        SOCKET SocketHandle = 0;

        if ( ( SocketHandle = accept( OwnerServer-&gt;GetSocketHandle(), (struct sockaddr*) &amp;ClientAddress, &amp;Length ) ) == INVALID_SOCKET )
        {
            throw ServerException( ""%s: accept failed"", __PRETTY_FUNCTION__ );
        }
        else
        {
            OnClientConnect( SocketHandle, ClientAddress );
        }

        FD_SET( SocketHandle, &amp;OwnerServer-&gt;FileDescriptorServer );
    }

    return ( SocketChangedCount &gt; 0 );
}

void ClientSockets::OnClientConnect( SOCKET ASocketHandle, struct sockaddr_in AClientAddress )
{
#if LOG_CONNECTION_AMOUNT &gt; 0
    static int ClientConnectionCounter = 0;

    if ( ClientConnectionCounter++ % LOG_CONNECTION_AMOUNT == 0 )
    {
        std::cout &lt;&lt; ""connect     fd: ["" &lt;&lt; ASocketHandle &lt;&lt; ""], serial: ["" &lt;&lt; ClientConnectionCounter &lt;&lt; ""], cnt: ["" &lt;&lt; Count() &lt;&lt; ""]"" &lt;&lt; std::endl;
    }
#endif

    ClientSocket *clientSocket = new ClientSocket( this, ASocketHandle );

    Append( clientSocket );
}

void ClientSockets::Read( void )
{
    for ( auto clientSocket : Items )
    {
        if ( FD_ISSET( clientSocket-&gt;GetSocketHandle(), &amp;FileDescriptorClient ) )
        {
            char buf[ SOCKETBUFLEN + 1 ];
            memset( buf, 0x0, sizeof( buf ) );

            int BytesReceived = recv( clientSocket-&gt;GetSocketHandle(), buf, SOCKETBUFLEN, 0 );

            if ( BytesReceived &gt; 0 )
            {
                std::string Message = ( std::string ) buf;

                if ( Message.substr( 0, 4 ) == ""quit"" )
                {
                    clientSocket-&gt;Close();
                    break;
                }
                else
                {
                    // ... do fancy stuff here...
                    std::cout &lt;&lt; ""read from socket: "" &lt;&lt; Message &lt;&lt; std::endl;
                }
            }
            else if ( BytesReceived == 0 )
            {
                clientSocket-&gt;Close();
                break;
            }
            else
            {
                throw ServerException( ""reading data failed"" );
            }
        }
    }
}



void ClientSockets::Remove( SOCKET ASocketHandle )
{
    FD_CLR( ASocketHandle, &amp; FileDescriptorClient );

    std::vector&lt; ClientSocket * &gt;::iterator it =
        Items.erase(
            std::remove_if(
                Items.begin(),
                Items.end(),
                [ ASocketHandle ]( Socket *sock ){ return sock-&gt;GetSocketHandle() == ASocketHandle; }
            )
        );

    delete *it;
}


void ClientSocket::Close( void )
{
#if LOG_CONNECTION_AMOUNT &gt; 0
    static int ClientDisconnectionCounter = 0;

    if ( ClientDisconnectionCounter++ % LOG_CONNECTION_AMOUNT == 0 )
    {
        std::cout &lt;&lt; ""disconnnect fd: ["" &lt;&lt; SocketHandle &lt;&lt; ""], serial: ["" &lt;&lt; ClientDisconnectionCounter &lt;&lt; ""], cnt: ["" &lt;&lt; FClientSockets-&gt;Count() &lt;&lt; ""]"" &lt;&lt; std::endl;
    }
#endif

    if ( closesocket( SocketHandle ) == SOCKET_ERROR )
    {
        throw ClientException( ""closing socket failed: %ld"", errno );
    }

    FClientSockets-&gt;Remove( SocketHandle );
}

ClientSocket::ClientSocket( ClientSockets *AClientSockets, SOCKET ASocketHandle ) :
    Socket( ASocketHandle ),
    FClientSockets( AClientSockets ) {}
</code></pre>

<p>The testing client application is very primitive: I used Embarcadero C++ Builder with a TClientSocket component which continuously toggles its connection state (in non-blocking-mode). 3 checkboxes are used to tell the application to run in loop mode (1), sleep before connect (2) and sleep before disconnect(3) and 1 TEdit is used to modify the sleep duration in Milliseconds.</p>

<p>The code of the cpp file is:</p>

<pre><code>#include &lt;vcl.h&gt;
#pragma hdrstop

#include ""UnitFormMain.h""

#pragma package(smart_init)
#pragma resource ""*.dfm""

TFormMain *FormMain;
//---------------------------------------------------------------------------
__fastcall TFormMain::TFormMain(TComponent* Owner)
    : TForm(Owner), Trigger( false ), sock( 0 )
{
}
//---------------------------------------------------------------------------
void __fastcall TFormMain::ClientSocket1Connect(TObject *Sender, TCustomWinSocket *Socket)

{
    static int ConnectionCounter = 0;

    if ( ConnectionCounter++ % 100 == 0 )
    {
        Memo1-&gt;Lines-&gt;Add( System::Sysutils::Format( ""connectionCounter '%d'"", ARRAYOFCONST(( ConnectionCounter )) ) );
    }

    if ( CheckBoxRunLoopComponent-&gt;Checked )
    {
        Trigger = true;
    }
}

//---------------------------------------------------------------------------
void __fastcall TFormMain::ClientSocket1Disconnect(TObject *Sender, TCustomWinSocket *Socket)
{

    if ( CheckBoxRunLoopComponent-&gt;Checked )
    {
        Trigger = true;
    }

}
//---------------------------------------------------------------------------
void __fastcall TFormMain::ButtonToggleConnectionComponentClick(TObject *Sender)
{
    ClientSocket1-&gt;Active = !ClientSocket1-&gt;Active;
}

//---------------------------------------------------------------------------
void __fastcall TFormMain::CheckBoxRunLoopComponentClick(TObject *Sender)
{
    if ( dynamic_cast&lt; TCheckBox * &gt;( Sender )-&gt;Checked )
    {

        for (;;)
        {
            Application-&gt;ProcessMessages();

            if ( !dynamic_cast&lt; TCheckBox * &gt;( Sender )-&gt;Checked )
            {
                break;
            }

            ToggleConnection();
        }
    }
}

void TFormMain::ToggleConnection( void )
{
    if ( Trigger )
    {
        Trigger = false;

        if ( ClientSocket1-&gt;Active )
        {
            if ( CheckBoxSleepOnDisconnect )
            {
                Sleep( StrToInt( Edit1-&gt;Text ) );
            }

            ClientSocket1-&gt;Close();

        }
        else
        {
            if ( CheckBoxSleepOnConnect-&gt;Checked )
            {
                Sleep( StrToInt( Edit1-&gt;Text ) );
            }

            ClientSocket1-&gt;Open();
        }
    }
}

//---------------------------------------------------------------------------
</code></pre>

<p>Way more interesting is the output of the test run (server):</p>

<p>Each 100th toggle is protocolled.
""cnt"" is the number of connections in the std::vector&lt;>.</p>

<p>The test started with 5 milliseconds sleep on each toggle. Somewhere around the 1500th toggling I decreased the sleeping time to 1 millisecond which lead into an immediate increase of active connections in the servers list. A few thousand connections later <code>select()</code> fails with <code>10038</code>.</p>

<pre><code>socket server fd '276' listening on 127.0.0.1:24442
void ServerSocket::Run()
socket changes: 1
connect     fd: [280], serial: [1], cnt: [0]
disconnnect fd: [280], serial: [1], cnt: [1]
connect     fd: [284], serial: [101], cnt: [0]
disconnnect fd: [284], serial: [101], cnt: [1]
connect     fd: [280], serial: [201], cnt: [0]
disconnnect fd: [280], serial: [201], cnt: [1]
connect     fd: [288], serial: [301], cnt: [0]
disconnnect fd: [288], serial: [301], cnt: [1]
connect     fd: [292], serial: [401], cnt: [0]
disconnnect fd: [292], serial: [401], cnt: [1]
connect     fd: [296], serial: [501], cnt: [0]
disconnnect fd: [296], serial: [501], cnt: [1]
connect     fd: [308], serial: [601], cnt: [0]
disconnnect fd: [308], serial: [601], cnt: [1]
connect     fd: [296], serial: [701], cnt: [0]
disconnnect fd: [296], serial: [701], cnt: [1]
connect     fd: [296], serial: [801], cnt: [0]
disconnnect fd: [296], serial: [801], cnt: [1]
connect     fd: [288], serial: [901], cnt: [0]
disconnnect fd: [288], serial: [901], cnt: [1]
connect     fd: [300], serial: [1001], cnt: [0]
disconnnect fd: [300], serial: [1001], cnt: [1]
connect     fd: [312], serial: [1101], cnt: [0]
disconnnect fd: [312], serial: [1101], cnt: [1]
connect     fd: [300], serial: [1201], cnt: [0]
disconnnect fd: [300], serial: [1201], cnt: [1]
connect     fd: [300], serial: [1301], cnt: [0]
disconnnect fd: [300], serial: [1301], cnt: [1]
connect     fd: [280], serial: [1401], cnt: [0]
disconnnect fd: [280], serial: [1401], cnt: [1]
connect     fd: [280], serial: [1501], cnt: [0]
disconnnect fd: [280], serial: [1501], cnt: [1]
connect     fd: [316], serial: [1601], cnt: [0]
socket changes: 2
disconnnect fd: [316], serial: [1601], cnt: [2]
socket changes: 1
socket changes: 2
socket changes: 1
socket changes: 2
socket changes: 1
connect     fd: [312], serial: [1701], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [312], serial: [1701], cnt: [4]
connect     fd: [324], serial: [1801], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [312], serial: [1801], cnt: [4]
connect     fd: [324], serial: [1901], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [324], serial: [1901], cnt: [4]
connect     fd: [320], serial: [2001], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [320], serial: [2001], cnt: [4]
socket changes: 2
socket changes: 1
connect     fd: [336], serial: [2101], cnt: [3]
disconnnect fd: [336], serial: [2101], cnt: [4]
connect     fd: [344], serial: [2201], cnt: [3]
disconnnect fd: [344], serial: [2201], cnt: [4]
connect     fd: [344], serial: [2301], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [336], serial: [2301], cnt: [4]
socket changes: 2
socket changes: 1
connect     fd: [344], serial: [2401], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [336], serial: [2401], cnt: [4]
connect     fd: [344], serial: [2501], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [344], serial: [2501], cnt: [4]
connect     fd: [344], serial: [2601], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [344], serial: [2601], cnt: [4]
connect     fd: [332], serial: [2701], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [332], serial: [2701], cnt: [4]
connect     fd: [344], serial: [2801], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [332], serial: [2801], cnt: [4]
socket changes: 2
socket changes: 1
connect     fd: [328], serial: [2901], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [324], serial: [2901], cnt: [4]
socket changes: 2
socket changes: 1
connect     fd: [328], serial: [3001], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [324], serial: [3001], cnt: [4]
socket changes: 2
socket changes: 1
connect     fd: [328], serial: [3101], cnt: [3]
socket changes: 2
socket changes: 1
disconnnect fd: [324], serial: [3101], cnt: [4]
connect     fd: [324], serial: [3201], cnt: [3]
disconnnect fd: [324], serial: [3201], cnt: [4]
connect     fd: [324], serial: [3301], cnt: [3]
disconnnect fd: [324], serial: [3301], cnt: [4]
socket changes: 2
socket changes: 1
error while server running: bool ClientSockets::HandleConnnections(): select failed, code: 10038
socket server fd '324' listening on 127.0.0.1:24442
void ServerSocket::Run()
socket changes: 0
</code></pre>

<p>In this protocol I also see that <code>select()</code> returns a value > 1 (""Socket changes: n"") which means that more than 1 socket changed its status. I think this might be the pointer to the correct solution, but right now I don't know how to handle this the right way.</p>

<p>Edit 3: Blocking/Noneblocking: In CPPBuilder you can use these design time components where you can set the properties at design time:</p>

<p><a href=""https://i.stack.imgur.com/5emLG.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/5emLG.png"" alt=""TClientSocktet property editor""></a></p>
",https://stackoverflow.com/questions/51058110/c-sockets-and-stress-testing,"['c++', 'sockets', 'testing', 'winsock2']",1727,TRUE,0,2
Stress Testing - EntityFramework issue - DBContext Creation,"<p>I am doing stress testing for web api (webapi2) with 20 users on boarding in 0 seconds. I am getting the following errors.</p>

<pre><code>System.Data.Entity.Core.EntityCommandExecutionException: An error occurred while executing the command definition. See the inner exception for details. ---&gt; System.InvalidOperationException: Invalid operation. The connection is closed.
</code></pre>

<p>Another error</p>

<pre><code>System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---&gt; System.InvalidOperationException: The connection was not closed. The connection's current state is connecting.
</code></pre>

<p>My code to get the DBContext, each time a new DBContext is getting created:</p>

<pre><code>public static ForcesChecker_Context GetDataContext()
    {           
        return new ForcesChecker_Context();            
    }
</code></pre>

<p>For one web api request this code is getting executed multiple times and multiple instances of this object is getting created. When I call 20 users at a time, it generates 20* ~10 = ~200 objects are created. </p>

<p>My connection string:</p>

<pre><code>Min Pool Size=1;Max Pool Size=200;
</code></pre>

<p>It seems there is a race condition.</p>

<p>What settings would help to allow more users to access my system concurrently?</p>
",https://stackoverflow.com/questions/48866133/stress-testing-entityframework-issue-dbcontext-creation,"['entity-framework-6', 'race-condition', 'stress-testing']",84,TRUE,0,1
Load/Stress testing an angular app,"<p>I have an angular application having some issues when more than a certain amount of users logs at the same time. I wish to load/stress test it with a realy simple scenario: user sees home page, logs in, then wait 30s, then logs out. I would like to have a 200-400 users crowd.</p>

<p>The problem is i can't find an appropriate tool to do the job. Is there any tester here which could help me ? Already tried Artillery and locust io.</p>

<p>Thank you already,</p>
",https://stackoverflow.com/questions/49854801/load-stress-testing-an-angular-app,"['angularjs', 'load-testing', 'stress-testing', 'locust', 'artillery']",4891,TRUE,0,1
Entity framework core stress testing is slow,"<ul>
<li><p>I build a .net core 2.1 application with EF core.</p>
</li>
<li><p>I have use Transaction with read uncommitted isolation level.</p>
</li>
<li><p>I build the async API and create a simple ef query async (get 5 fields of first user, not reference to other table).
[query user][1]</p>
</li>
<li><p>When i create a single request, the query take small time</p>
</li>
<li><p>When i stress test with 10 threads, ramp-up: 5, loop forever (using jmeter), the query time is same</p>
</li>
<li><p>However, when i stress test to the api using jmeter (100 threads, ramp-up: 20s, loop forever), some query take small time, some query take large time (maybe 5s, 10s, 25s ...), another query throw connection timeout exception</p>
</li>
<li><p>what should i do?</p>
</li>
</ul>
<blockquote>
<p><strong>Issue resolved</strong>: Take some days to investigating, i tried with this solution and it's working well. So, i will share it on this post, if you have other solutions to increase the performance, pls tell me about it.</p>
</blockquote>
<ul>
<li><p>Creating database connections is an expensive process that takes time. You can specify that you want a minimum pool of connections that should be created and kept open for the lifetime of the application. These are then reused for each database call.</p>
</li>
<li><p>Should use transaction isolation level &quot;Read Uncommitted&quot;</p>
</li>
<li><p>Should use the same Database Connection for multiple operations on one request</p>
</li>
<li><p>All APIs, methods should be Async method, make sure do not mixing Async with Sync.</p>
<p>Thanks all !!!</p>
</li>
</ul>
",https://stackoverflow.com/questions/52466482/entity-framework-core-stress-testing-is-slow,"['multithreading', 'asp.net-core', 'jmeter', 'stress-testing', 'entity-framework-core-2.1']",699,FALSE,0,1
How to create stress testing for Azure mobile app?,"<p>I create Azure mobile app for sync data from devices. I created this backend  on ApiController(post, get, delete, put requests). My app use azure sql database and cosmos db. 
  How to create stress testing for my Azure mobile app? 
  I found this <a href=""https://learn.microsoft.com/en-us/vsts/load-test/app-service-web-app-performance-test"" rel=""nofollow noreferrer"">article</a>. but it looks too general and simply. I want to check my controllers and databases under load. Is this possible?
  I do not have an UI in my backend, the client app sends data to my controllers. How can I do it in test?</p>
",https://stackoverflow.com/questions/48421039/how-to-create-stress-testing-for-azure-mobile-app,"['azure', 'azure-sql-database', 'azure-mobile-services', 'backend']",177,TRUE,0,2
"Response time, Load Testing and Stress testing with Rest Assured","<p>I know there is a way to measure the response time of a Rest API with Rest Assured. But can we increase the user load and measure the response time using Rest assured ,or is there any other suggested way to count in those parameters (Load / Stress) using API Testing. TIA</p>
",https://stackoverflow.com/questions/50672118/response-time-load-testing-and-stress-testing-with-rest-assured,"['performance-testing', 'load-testing', 'rest-assured', 'web-api-testing']",4385,TRUE,0,1
Stress testing with an external API server in Java Spring - mock response?,"<p>I have a microservice architecture setup and I am looking to stress test it. I have quite a complex setup with some synchronous http requests and a message queue system (RabbitMQ).</p>

<p>As I am making use of the API server of another company, I do not want to run the risk of stressing their server. This request is a synchronous call which asks the API provider and returns quite a sizable response. Instead, I want to have some stub/mock which will return a similar response and with similar latency. Say, for example, a response from this server is 5s and returns 2kb of data, then I want to introduce a 5s delay and return more or less some comparatively sized data (in other words I want some customization) in that format which I have prepared.</p>

<p>To get to this point, I require (a) to capture a sizable response and edit it. Particularly, I want to edit it to filter out some sensitive data and also to increase its size and to make it larger so that I can stress the bandwidth aspect of the system. (b) I require to stay as close as possible to the native solution to get a good grasp of response times from this end and also I want as minimal overhead as possible when stress testing. I have been looking at some solutions like WireMock which, if my understanding is correct, set up a fake server to test against. This won't work for me as I require to use my own servers (which are TomCat based) instead to stay as close as possible to my architecture.</p>
",https://stackoverflow.com/questions/53731815/stress-testing-with-an-external-api-server-in-java-spring-mock-response,"['java', 'spring', 'microservices', 'stress-testing', 'wiremock']",1114,TRUE,1,2
How to generate huge dynamic test data using jmeter to perform stress testing,"<p>I need to preform stress testing on Sign Up functionality, where 1000 user should login into the application at same time. It's very difficult to create 1000 test data every time manually in excel sheet and parse to jmeter. Can any one help me out where and how to create script for generating unique test data dynamically using jmeter. </p>
",https://stackoverflow.com/questions/50441035/how-to-generate-huge-dynamic-test-data-using-jmeter-to-perform-stress-testing,"['jmeter', 'stress-testing', 'jmeter-3.2', 'jmeter-4.0']",2632,TRUE,0,1
k6 simulate logged in wordpress user,"<p>I am trying to simulate a logged in user activity for stress testing a WordPress site using this script:</p>

<pre><code>import http from ""k6/http"";
import { sleep } from ""k6"";

export default function() {
  var url = ""http://example.com.com/wp-login.php"";
  var payload = JSON.stringify({ user_login: ""admin"", user_pass: ""adminpass"" });
  var params =  { headers: { ""Content-Type"": ""application/json"" } }
  http.post(url, payload, params);

  console.log(payload);

  http.get(""http://example.com/wp-admin/post-new.php"")

  sleep(1)
};
</code></pre>

<p>This doesn't work and I am redirected to the login page.
What am I doing wrong?</p>
",https://stackoverflow.com/questions/51548493/k6-simulate-logged-in-wordpress-user,"['wordpress', 'k6']",591,TRUE,2,1
Why should I use max-request to avoid memory leaks?,"<p>Why should I use max-request to avoid memory leaks?</p>

<p>I have stress testing my application and memory leak has occurred, and max-request is set so memory leak is gone. Is this a required option? Or is it my fault?</p>
",https://stackoverflow.com/questions/51586463/why-should-i-use-max-request-to-avoid-memory-leaks,['uwsgi'],2243,TRUE,2,1
How to profile a stress test for a distributed system in Python?,"<p>First, I have looked into other threads but could not find a single tool (all in one) for profiling performance.
I want to profile system performance while doing some stress testing. 
The system is written into many .py files and have many processes running at the same time. Mainly interested in CPU, Memory, Time and Throughput.  </p>
",https://stackoverflow.com/questions/53833242/how-to-profile-a-stress-test-for-a-distributed-system-in-python,"['python', 'python-3.x']",96,FALSE,1,0
How to write a method in Ruby that increments a passed in string by X amount of random characters,"<p>I am new to ruby. I vaguely remember utilizing incremental functions in Java, e.g.</p>

<pre><code>for i = 0; i &lt; 100; i++
</code></pre>

<p>I am stress testing a text box by adding a LOT of characters to the end of a sentence to see if the text box works under such circumstances. .succ is not helping me. im trying to add 100 randomized characters to the end of a string.</p>
",https://stackoverflow.com/questions/53963330/how-to-write-a-method-in-ruby-that-increments-a-passed-in-string-by-x-amount-of,['ruby'],83,TRUE,-1,1
Python read from text file error,"<p>I'm writing a scrip for stress testing, the file i'm reading from is my input and it's a text file. Ideally I will be iterating the list of address to perform an stress test. The problem stems with the input, it's inefficient as a space won't be processed. </p>

<p>I'm having an odd glitch, my output works but it add odd spaces.</p>

<p>I've tried readlines(), but it yields the same result.</p>

<pre><code>file = open('ip.txt', 'r') 
for line in file:
    print (line)

// Current OUTPUT
&gt;&gt; X.X.X.X
&gt;&gt; 
&gt;&gt; Y.Y.Y.Y
&gt;&gt; 

// Desired OUTPUT
&gt;&gt; OUTPUT:
&gt;&gt; X.X.X.X
&gt;&gt; Y.Y.Y.Y
</code></pre>

<p>thank You,</p>
",https://stackoverflow.com/questions/50276661/python-read-from-text-file-error,['python-2.7'],861,TRUE,0,2
Loadrunner - How do I simulate quitting or closing a browser-session in Web HTTP/HTML protocol?,"<p>I'm stress-testing an application that only allows users to exit the application in one of two ways:
 1. Quitting the open browser-session.
 2. Allowing 5 minutes of idle-time to elapse.</p>

<p>I'd like to be able to write syntax in the Loadrunner script that quits the browser session. The protocol I would like to use is Web HTTP/HTML.</p>

<p>Does anyone know how to do this, or have any advice?</p>

<p>Thank you!</p>
",https://stackoverflow.com/questions/50395351/loadrunner-how-do-i-simulate-quitting-or-closing-a-browser-session-in-web-http,"['loadrunner', 'stress-testing']",357,FALSE,0,1
Find out which process opens a bunch of mysterious connections in Linux (Azure),"<p>When stress testing our server I see a lot of (200+) open HTTPS connections to mysterious ip in Ubuntu 16.04 running in Azure. This ip belongs to Microsoft.</p>

<ul>
<li>This ip does not belong to any of our machines nor the services we're
using</li>
<li>When shutting down the server process, connections still keep appearing but only 4-5 at a time compared to 200+ when stress testing and server process enabled. Server is not running any other services (fresh install from Azure + our process)</li>
<li>No matter what I try to do, I cannot see the process where these connections are coming from (netstat, lsof, etc. running as root)</li>
</ul>

<p>netstat prints a lot of these</p>

<pre><code>$ sudo netstat -p | grep 52.239.137.70
tcp        0      0 10.0.2.6:59202          52.239.137.70:https     TIME_WAIT   -
...
</code></pre>

<p>When trying to open this ip in a web browser (HTTP or HTTPS), you see this error</p>

<pre><code>&lt;error xmlns=""http://schemas.microsoft.com/ado/2007/08/dataservices/metadata""&gt;
    &lt;code&gt;InvalidUri&lt;/code&gt;
    &lt;message xml:lang=""en-US""&gt;
        The requested URI does not represent any resource on the server. RequestId...
    &lt;/message&gt;
&lt;/error&gt;
</code></pre>

<p>So I'm guessing this is somehow connected to Azure Blob service, since when googling the error message you get a bunch forum posts troubleshooting blobs.</p>

<p>I have troubleshooted this for two days and cannot find an answer. I am worried that this will cause problems when under high load in production because when testing with small amounts of clients you still see hundreds of these mystery connections.</p>
",https://stackoverflow.com/questions/53065331/find-out-which-process-opens-a-bunch-of-mysterious-connections-in-linux-azure,"['linux', 'azure', 'connection']",43,TRUE,0,1
Is it possible to create a &gt;8 EiB sparse file anywhere?,"<p>Background: as part of trying to ensure complete coverage in my unit tests, I'm stress-testing some overflow checks.</p>

<p>So, I tried to create a 16 EiB sparse file. </p>

<p>However, on Linux/Btrfs this seems to fail at 8 EiB (2^63 bytes) rather than 16 EiB (2^64 bytes), likely because <code>off_t</code> is signed.</p>

<p>Is there any OS/filesystem combo which <em>actually</em>, rather than theoretically, allows creating such files?</p>

<p>(-1 omitted in each of the numbers above)</p>

<hr>

<p>This works:</p>

<pre><code>dd if=/dev/zero of=sparse_file bs=1 count=0 seek=9223372036854775807
</code></pre>

<p>This fails:</p>

<pre><code>dd if=/dev/zero of=sparse_file bs=1 count=0 seek=9223372036854775808
</code></pre>
",https://stackoverflow.com/questions/52360077/is-it-possible-to-create-a-8-eib-sparse-file-anywhere,"['filesystems', 'limit', 'sparse-file']",25,FALSE,2,0
Timeouts and slow requests to Redis when using StackExchange.Redis library,"<p>We've started using Redis in our application to persist API responses there. I chose to use StackExchange.Redis to work with it and after implementing the whole logic of saving the responses the results of stress testing became quite disappointing. The total throughput of the application degraded in 3-4 times! It was about 550-600rps and now it is around 180rps (and can be even slower). </p>

<p>Sometimes there are no exceptions at all during a stress testing process but from time to time Timeout exceptions occur like these ones:</p>

<pre><code>Timeout awaiting response (5094ms elapsed, timeout is 5000ms), inst: 0, qs: 10, in: 65536, mgr: 10 of 10 available, IOCP: (Busy=26,Free=974,Min=8,Max=1000), WORKER: (Busy=3,Free=32764,Min=8,Max=32767), v: 2.0.513.63329 
Timeout awaiting response (5063ms elapsed, timeout is 5000ms), inst: 0, qs: 4, in: 47355, mgr: 10 of 10 available, IOCP: (Busy=40,Free=960,Min=8,Max=1000), WORKER: (Busy=34,Free=32733,Min=8,Max=32767), v: 2.0.513.63329 
The timeout was reached before the message could be written to the output buffer, and it was not sent (5000ms, inst=3, qs=3, in=10, active=HMSET), inst: 3, qs: 3, in: 10, mgr: 10 of 10 available, IOCP: (Busy=36,Free=964,Min=8,Max=1000), WORKER: (Busy=34,Free=32733,Min=8,Max=32767), v: 2.0.513.63329 
</code></pre>

<p>And here is the total overview picture of all timeouts exceptions occurred during the last stress test:
<a href=""https://i.stack.imgur.com/rTtZt.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/rTtZt.png"" alt=""Statistics""></a></p>

<p>I've tried to implement connection pooling (based on TotalOutstanding property), tried to decrease the number of requests to Redis and so on but that didn't help. </p>

<p>I executed SHOW LOG and LATENCY DOCTOR commands but it seems that it's all okay with our Redis instance (although we've disabled THP as it was recommended by the doctor).</p>

<p>My assumption is that some very big API responses prevent other requests from completing. I think so because of high inbound traffic values. Is it correct? What can be done with that? Do I need to separate requests somehow?</p>
",https://stackoverflow.com/questions/53601215/timeouts-and-slow-requests-to-redis-when-using-stackexchange-redis-library,"['c#', 'asp.net-core', 'redis', 'timeout', 'stackexchange.redis']",3562,FALSE,2,1
call limited 32 calls on pjsip max_calls,"<p>We are having a problem with the max_calls settings on PJSIP and Asterisk.
We are stress-testing our Asterisk server, but found that we had a max 32 active call limitation on our PJSIP module. We are using PJSIP to test our Asterisk server</p>

<p>After a quick google we found the following settings would fix the problem.</p>

<p><div class=""snippet"" data-lang=""js"" data-hide=""false"" data-console=""true"" data-babel=""false"">
<div class=""snippet-code"">
<pre class=""snippet-code-html lang-html prettyprint-override""><code>Following steps can be taken to increase number of calls supported on PJSIP:
Example: If you have to increase simultaneous calls to 1000 change the following:
1.       Change PJSUA_MAX_CALLS to 1000 and PJSUA_MAX_ACC to 1000
2.       Change PJ_IOQUEUE_MAX_HANDLES to 2000 (double of desired number of calls).
3.       Change __FD_SETSIZE to double to 2000 (double of desired number of calls).
4.       Change PJSUA_MAX_PLAYERS to 1000.
5.       Recompile pjsip using following steps:
a.       ./configure --disable-ssl --disable-sound; 
b.      make dep
c.       make 
d.      make install
6.       Recompile your application with new libs.</code></pre>
</div>
</div>
</p>

<p>Somehow This is not working for us;
What are we doing wrong here?
Anybody suggestions. Help would be higly appreciated. </p>

<p>Our code in the config_site.php file</p>

<p><div class=""snippet"" data-lang=""js"" data-hide=""false"" data-console=""true"" data-babel=""false"">
<div class=""snippet-code"">
<pre class=""snippet-code-html lang-html prettyprint-override""><code>/*
 * This file contains several sample settings especially for Windows
 * Mobile and Symbian targets. You can include this file in your
 * &lt;pj/config_site.h&gt; file.
 *
 * The Windows Mobile and Symbian settings will be activated
 * automatically if you include this file.
 *
 * In addition, you may specify one of these macros (before including
 * this file) to activate additional settings:
 *
 * #define PJ_CONFIG_NOKIA_APS_DIRECT
 *   Use this macro to activate the APS-Direct feature. Please see
 *   http://trac.pjsip.org/repos/wiki/Nokia_APS_VAS_Direct for more
 *   info.
 *
 * #define PJ_CONFIG_WIN32_WMME_DIRECT
 *   Configuration to activate ""APS-Direct"" media mode on Windows or
 *   Windows Mobile, useful for testing purposes only.
 */


/*
 * Typical configuration for WinCE target.
 */
#if defined(PJ_WIN32_WINCE) &amp;&amp; PJ_WIN32_WINCE!=0

    /*
     * PJLIB settings.
     */

    /* Disable floating point support */
    #define PJ_HAS_FLOATING_POINT               0

    /*
     * PJMEDIA settings
     */

    /* Select codecs to disable */
    #define PJMEDIA_HAS_L16_CODEC               0
    #define PJMEDIA_HAS_ILBC_CODEC              0

    /* We probably need more buffers on WM, so increase the limit */
    #define PJMEDIA_SOUND_BUFFER_COUNT          32

    /* Fine tune Speex's default settings for best performance/quality */
    #define PJMEDIA_CODEC_SPEEX_DEFAULT_QUALITY 5

    /* For CPU reason, disable speex AEC and use the echo suppressor. */
    #define PJMEDIA_HAS_SPEEX_AEC               0

    /* Previously, resampling is disabled due to performance reason and
     * this condition prevented some 'light' wideband codecs (e.g: G722.1)
     * to work along with narrowband codecs. Lately, some tests showed
     * that 16kHz &lt;-&gt; 8kHz resampling using libresample small filter was
     * affordable on ARM9 260 MHz, so here we decided to enable resampling.
     * Note that it is important to make sure that libresample is created
     * using small filter. For example PJSUA_DEFAULT_CODEC_QUALITY must
     * be set to 3 or 4 so pjsua-lib will apply small filter resampling.
     */
    //#define PJMEDIA_RESAMPLE_IMP              PJMEDIA_RESAMPLE_NONE
    #define PJMEDIA_RESAMPLE_IMP                PJMEDIA_RESAMPLE_LIBRESAMPLE

    /* Use the lighter WSOLA implementation */
    #define PJMEDIA_WSOLA_IMP                   PJMEDIA_WSOLA_IMP_WSOLA_LITE

    /*
     * PJSIP settings.
     */

    /* Set maximum number of dialog/transaction/calls to minimum to reduce
     * memory usage
     */
    #define PJSIP_MAX_TSX_COUNT                 31
    #define PJSIP_MAX_DIALOG_COUNT              31
    #define PJSUA_MAX_CALLS                     64

    /*
     * PJSUA settings
     */

    /* Default codec quality, previously was set to 5, however it is now
     * set to 4 to make sure pjsua instantiates resampler with small filter.
     */
    #define PJSUA_DEFAULT_CODEC_QUALITY         4

    /* Set maximum number of objects to minimum to reduce memory usage */
    #define PJSUA_MAX_ACC                       64
    #define PJSUA_MAX_PLAYERS                   64
    #define PJSUA_MAX_RECORDERS                 4
    #define PJSUA_MAX_CONF_PORTS                (PJSUA_MAX_CALLS+2*PJSUA_MAX_PLAYERS)
    #define PJSUA_MAX_BUDDIES                   32

#endif  /* PJ_WIN32_WINCE */


/*
 * Typical configuration for Symbian OS target
 */
#if defined(PJ_SYMBIAN) &amp;&amp; PJ_SYMBIAN!=0

    /*
     * PJLIB settings.
     */

    /* Disable floating point support */
    #define PJ_HAS_FLOATING_POINT               0

    /* Misc PJLIB setting */
    #define PJ_MAXPATH                          80

    /* This is important for Symbian. Symbian lacks vsnprintf(), so
     * if the log buffer is not long enough it's possible that
     * large incoming packet will corrupt memory when the log tries
     * to log the packet.
     */
    #define PJ_LOG_MAX_SIZE                     (PJSIP_MAX_PKT_LEN+500)

    /* Since we don't have threads, log buffer can use static buffer
     * rather than stack
     */
    #define PJ_LOG_USE_STACK_BUFFER             0

    /* Disable check stack since it increases footprint */
    #define PJ_OS_HAS_CHECK_STACK               0


    /*
     * PJMEDIA settings
     */

    /* Disable non-Symbian audio devices */
    #define PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO     0
    #define PJMEDIA_AUDIO_DEV_HAS_WMME          0

    /* Select codecs to disable */
    #define PJMEDIA_HAS_L16_CODEC               0
    #define PJMEDIA_HAS_ILBC_CODEC              0
    #define PJMEDIA_HAS_G722_CODEC              0

    /* Fine tune Speex's default settings for best performance/quality */
    #define PJMEDIA_CODEC_SPEEX_DEFAULT_QUALITY 5

    /* For CPU reason, disable speex AEC and use the echo suppressor. */
    #define PJMEDIA_HAS_SPEEX_AEC               0

    /* Previously, resampling is disabled due to performance reason and
     * this condition prevented some 'light' wideband codecs (e.g: G722.1)
     * to work along with narrowband codecs. Lately, some tests showed
     * that 16kHz &lt;-&gt; 8kHz resampling using libresample small filter was
     * affordable on ARM9 222 MHz, so here we decided to enable resampling.
     * Note that it is important to make sure that libresample is created
     * using small filter. For example PJSUA_DEFAULT_CODEC_QUALITY must
     * be set to 3 or 4 so pjsua-lib will apply small filter resampling.
     */
    //#define PJMEDIA_RESAMPLE_IMP              PJMEDIA_RESAMPLE_NONE
    #define PJMEDIA_RESAMPLE_IMP                PJMEDIA_RESAMPLE_LIBRESAMPLE

    /* Use the lighter WSOLA implementation */
    #define PJMEDIA_WSOLA_IMP                   PJMEDIA_WSOLA_IMP_WSOLA_LITE

    /* We probably need more buffers especially if MDA audio backend
     * is used, so increase the limit
     */
    #define PJMEDIA_SOUND_BUFFER_COUNT          32

    /*
     * PJSIP settings.
     */

    /* Disable safe module access, since we don't use multithreading */
    #define PJSIP_SAFE_MODULE                   0

    /* Use large enough packet size  */
    #define PJSIP_MAX_PKT_LEN                   2000

    /* Symbian has problem with too many large blocks */
    #define PJSIP_POOL_LEN_ENDPT                1000
    #define PJSIP_POOL_INC_ENDPT                1000
    #define PJSIP_POOL_RDATA_LEN                2000
    #define PJSIP_POOL_RDATA_INC                2000
    #define PJSIP_POOL_LEN_TDATA                2000
    #define PJSIP_POOL_INC_TDATA                512
    #define PJSIP_POOL_LEN_UA                   2000
    #define PJSIP_POOL_INC_UA                   1000
    #define PJSIP_POOL_TSX_LAYER_LEN            256
    #define PJSIP_POOL_TSX_LAYER_INC            256
    #define PJSIP_POOL_TSX_LEN                  512
    #define PJSIP_POOL_TSX_INC                  128

    /*
     * PJSUA settings.
     */

    /* Default codec quality, previously was set to 5, however it is now
     * set to 4 to make sure pjsua instantiates resampler with small filter.
     */
    #define PJSUA_DEFAULT_CODEC_QUALITY         4

    /* Set maximum number of dialog/transaction/calls to minimum */
    #define PJSIP_MAX_TSX_COUNT                 31
    #define PJSIP_MAX_DIALOG_COUNT              31
    #define PJSUA_MAX_CALLS                     64

    /* Other pjsua settings */
    #define PJSUA_MAX_ACC                       64
    #define PJSUA_MAX_PLAYERS                   64
    #define PJSUA_MAX_RECORDERS                 4
    #define PJSUA_MAX_CONF_PORTS                (PJSUA_MAX_CALLS+2*PJSUA_MAX_PLAYERS)
    #define PJSUA_MAX_BUDDIES                   32
#endif


/*
 * Additional configuration to activate APS-Direct feature for
 * Nokia S60 target
 *
 * Please see http://trac.pjsip.org/repos/wiki/Nokia_APS_VAS_Direct
 */
#ifdef PJ_CONFIG_NOKIA_APS_DIRECT

    /* MUST use switchboard rather than the conference bridge */
    #define PJMEDIA_CONF_USE_SWITCH_BOARD       1

    /* Enable APS sound device backend and disable MDA &amp; VAS */
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_MDA      0
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_APS      1
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_VAS      0

    /* Enable passthrough codec framework */
    #define PJMEDIA_HAS_PASSTHROUGH_CODECS      1

    /* And selectively enable which codecs are supported by the handset */
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMU  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMA  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_AMR   1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_G729  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_ILBC  1

#endif


/*
 * Additional configuration to activate VAS-Direct feature for
 * Nokia S60 target
 *
 * Please see http://trac.pjsip.org/repos/wiki/Nokia_APS_VAS_Direct
 */
#ifdef PJ_CONFIG_NOKIA_VAS_DIRECT

    /* MUST use switchboard rather than the conference bridge */
    #define PJMEDIA_CONF_USE_SWITCH_BOARD       1

    /* Enable VAS sound device backend and disable MDA &amp; APS */
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_MDA      0
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_APS      0
    #define PJMEDIA_AUDIO_DEV_HAS_SYMB_VAS      1

    /* Enable passthrough codec framework */
    #define PJMEDIA_HAS_PASSTHROUGH_CODECS      1

    /* And selectively enable which codecs are supported by the handset */
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMU  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMA  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_AMR   1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_G729  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_ILBC  1

#endif


/*
 * Configuration to activate ""APS-Direct"" media mode on Windows,
 * useful for testing purposes only.
 */
#ifdef PJ_CONFIG_WIN32_WMME_DIRECT

    /* MUST use switchboard rather than the conference bridge */
    #define PJMEDIA_CONF_USE_SWITCH_BOARD       1

    /* Only WMME supports the ""direct"" feature */
    #define PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO     0
    #define PJMEDIA_AUDIO_DEV_HAS_WMME          1

    /* Enable passthrough codec framework */
    #define PJMEDIA_HAS_PASSTHROUGH_CODECS      1

    /* Only PCMA and PCMU are supported by WMME-direct */
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMU  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMA  1
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_AMR   0
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_G729  0
    #define PJMEDIA_HAS_PASSTHROUGH_CODEC_ILBC  0

#endif

/*
 * iPhone sample settings.
 */
#if PJ_CONFIG_IPHONE
    /*
     * PJLIB settings.
     */

    /* Both armv6 and armv7 has FP hardware support.
     * See https://trac.pjsip.org/repos/ticket/1589 for more info
     */
    #define PJ_HAS_FLOATING_POINT               1

    /*
     * PJMEDIA settings
     */

    /* We have our own native CoreAudio backend */
    #define PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO     0
    #define PJMEDIA_AUDIO_DEV_HAS_WMME          0
    #define PJMEDIA_AUDIO_DEV_HAS_COREAUDIO     1

    /* The CoreAudio backend has built-in echo canceller! */
    #define PJMEDIA_HAS_SPEEX_AEC    0

    /* Disable some codecs */
    #define PJMEDIA_HAS_L16_CODEC               0
    #define PJMEDIA_HAS_G722_CODEC              0

    /* Use the built-in CoreAudio's iLBC codec (yay!) */
    #define PJMEDIA_HAS_ILBC_CODEC              1
    #define PJMEDIA_ILBC_CODEC_USE_COREAUDIO    1

    /* Fine tune Speex's default settings for best performance/quality */
    #define PJMEDIA_CODEC_SPEEX_DEFAULT_QUALITY 5

    /*
     * PJSIP settings.
     */

    /* Increase allowable packet size, just in case */
    //#define PJSIP_MAX_PKT_LEN                 2000

    /*
     * PJSUA settings.
     */

    /* Default codec quality, previously was set to 5, however it is now
     * set to 4 to make sure pjsua instantiates resampler with small filter.
     */
    #define PJSUA_DEFAULT_CODEC_QUALITY         4

    /* Set maximum number of dialog/transaction/calls to minimum */
    #define PJSIP_MAX_TSX_COUNT                 31
    #define PJSIP_MAX_DIALOG_COUNT              31
    #define PJSUA_MAX_CALLS                     64

    /* Other pjsua settings */
    #define PJSUA_MAX_ACC                       64
    #define PJSUA_MAX_PLAYERS                   64
    #define PJSUA_MAX_RECORDERS                 4
    #define PJSUA_MAX_CONF_PORTS                (PJSUA_MAX_CALLS+2*PJSUA_MAX_PLAYERS)
    #define PJSUA_MAX_BUDDIES                   32

#endif

/*
 * Android sample settings.
 */
#if PJ_CONFIG_ANDROID

    #define PJ_ANDROID                          1

    /*
     * PJLIB settings.
     */

    /* Disable floating point support */
    #undef PJ_HAS_FLOATING_POINT
    #define PJ_HAS_FLOATING_POINT               0

    /*
     * PJMEDIA settings
     */

    /* We have our own OpenSL ES backend */
    #define PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO     0
    #define PJMEDIA_AUDIO_DEV_HAS_WMME          0
    #define PJMEDIA_AUDIO_DEV_HAS_OPENSL        0
    #define PJMEDIA_AUDIO_DEV_HAS_ANDROID_JNI   1

    /* Disable some codecs */
    #define PJMEDIA_HAS_L16_CODEC               0
    #define PJMEDIA_HAS_G722_CODEC              0

    /* Fine tune Speex's default settings for best performance/quality */
    #define PJMEDIA_CODEC_SPEEX_DEFAULT_QUALITY 5

    /*
     * PJSIP settings.
     */

    /* Increase allowable packet size, just in case */
    //#define PJSIP_MAX_PKT_LEN                 2000

    /*
     * PJSUA settings.
     */

    /* Default codec quality, previously was set to 5, however it is now
     * set to 4 to make sure pjsua instantiates resampler with small filter.
     */
    #define PJSUA_DEFAULT_CODEC_QUALITY         4

    /* Set maximum number of dialog/transaction/calls to minimum */
    #define PJSIP_MAX_TSX_COUNT                 31
    #define PJSIP_MAX_DIALOG_COUNT              31
    #define PJSUA_MAX_CALLS                     64

    /* Other pjsua settings */
    #define PJSUA_MAX_ACC                       64
    #define PJSUA_MAX_PLAYERS                   64
    #define PJSUA_MAX_RECORDERS                 4
    #define PJSUA_MAX_CONF_PORTS                (PJSUA_MAX_CALLS+2*PJSUA_MAX_PLAYERS)
    #define PJSUA_MAX_BUDDIES                   32
#endif


/*
 * BB10
 */
#if defined(PJ_CONFIG_BB10) &amp;&amp; PJ_CONFIG_BB10
    /* Quality 3 - 4 to use resampling small filter */
    #define PJSUA_DEFAULT_CODEC_QUALITY                 4
    #define PJMEDIA_HAS_LEGACY_SOUND_API                0
    #undef PJMEDIA_HAS_SPEEX_AEC
    #define PJMEDIA_HAS_SPEEX_AEC                       0
    #undef PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO
    #define PJMEDIA_AUDIO_DEV_HAS_PORTAUDIO             0
#endif


/*
 * Minimum size
 */
#ifdef PJ_CONFIG_MINIMAL_SIZE

#   undef PJ_OS_HAS_CHECK_STACK
#   define PJ_OS_HAS_CHECK_STACK        0
#   define PJ_LOG_MAX_LEVEL             0
#   define PJ_ENABLE_EXTRA_CHECK        0
#   define PJ_HAS_ERROR_STRING          0
#   undef PJ_IOQUEUE_MAX_HANDLES
/* Putting max handles to lower than 32 will make pj_fd_set_t size smaller
 * than native fdset_t and will trigger assertion on sock_select.c.
 */
#   define PJ_IOQUEUE_MAX_HANDLES       128
#   define PJ_CRC32_HAS_TABLES          0
#   define PJSIP_MAX_TSX_COUNT          15
#   define PJSIP_MAX_DIALOG_COUNT       15
#   define PJSIP_UDP_SO_SNDBUF_SIZE     4000
#   define PJSIP_UDP_SO_RCVBUF_SIZE     4000
#   define PJMEDIA_HAS_ALAW_ULAW_TABLE  0

#elif defined(PJ_CONFIG_MAXIMUM_SPEED)
#   define PJ_SCANNER_USE_BITWISE       0
#   undef PJ_OS_HAS_CHECK_STACK
#   define PJ_OS_HAS_CHECK_STACK        0
#   define PJ_LOG_MAX_LEVEL             3
#   define PJ_ENABLE_EXTRA_CHECK        0
#   define PJ_IOQUEUE_MAX_HANDLES       5000
#   define PJSIP_MAX_TSX_COUNT          ((640*1024)-1)
#   define PJSIP_MAX_DIALOG_COUNT       ((640*1024)-1)
#   define PJSIP_UDP_SO_SNDBUF_SIZE     (24*1024*1024)
#   define PJSIP_UDP_SO_RCVBUF_SIZE     (24*1024*1024)
#   define PJ_DEBUG                     0
#   define PJSIP_SAFE_MODULE            0
#   define PJ_HAS_STRICMP_ALNUM         0
#   define PJ_HASH_USE_OWN_TOLOWER      1
#   define PJSIP_UNESCAPE_IN_PLACE      1

#   if defined(PJ_WIN32) || defined(PJ_WIN64)
#     define PJSIP_MAX_NET_EVENTS       10
#   endif

#   define PJSUA_MAX_CALLS              512

#endif</code></pre>
</div>
</div>
</p>
",https://stackoverflow.com/questions/50442195/call-limited-32-calls-on-pjsip-max-calls,"['asterisk', 'pjsip']",1643,FALSE,0,1
"&quot;status&quot;:500,&quot;error&quot;:&quot;Internal Server Error&quot;,&quot;exception&quot;:&quot;com.netflix.zuul.exception.ZuulException&quot;,&quot;message&quot;:&quot;REJECTED_THREAD_EXECUTION&quot;}","<p>{""timestamp"":1531026130373,""status"":500,""error"":""Internal Server Error"",""exception"":""com.netflix.zuul.exception.ZuulException"",""message"":""REJECTED_THREAD_EXECUTION""}</p>

<p>I am getting above exception while running stress testing on my one of the API redirecting from zull gateway.</p>

<p>Here is some of the more points which I want to mentioned here.</p>

<ol>
<li>I am using netflix zull and eureka for discovery my microservices (spring boot).</li>
<li>Everything is working fine from my angular application, postman and swagger.</li>
<li>Now I am trying to do stress testing using Jmeter tool which is working fine for 10 users load. But as soon as I am mentioning more than 10 no. of thread users. It is giving me above mentioned exception. </li>
<li>I tried to replicate same scenario on my local laptop and used the same jmeter. And this time rather giving the server host name I try to test with my laptop hostname (all code and services are same). I mentioned 10+ no. of thread users in jmeter and that is working fine with no above exception. </li>
</ol>

<p>I read lot of stack-overflow posts, github links and other web search for this issue but none of solution seems to working. I tried and some other</p>

<p>ribbon:
  ReadTimeout: 60000
  ConnectTimeout: 60000</p>

<p>hystrix:
  command:
    default:
      execution:
        isolation:
          strategy: THREAD
          thread:
            timeoutInMilliseconds: 60000</p>
",https://stackoverflow.com/questions/51229203/status500-errorinternal-server-error-exceptioncom-netflix-zuul-excep,"['spring-boot', 'jmeter', 'microservices', 'spring-cloud', 'netflix']",2681,FALSE,1,1
Basic Python Algorithm Stress Test,"<p>Hi I am learning the basics of algorithms using python, this is my first stress testing script (I am still new to python).</p>

<p>When i run the test both the variables fast and result are printed as 0, the random number generator i included doesn't seem to be creating numbers for the functions mpp and mppf to use, either that or they are not assigning their results to the relevant variable, hence the variable remains = 0 and the loop keeps printing '0 0 OK'</p>

<p>I am receiving no errors except for the fact that my script isn't doing what i want it too!</p>

<pre><code>  import random
    result = 0
    fast = 0

    while result == fast:
        if __name__ == '__main__':
            n = (random.randint(2, 11))
            a = list(random.randint(0, 99999) for r in range(n))
            assert (len(a) == n)

            def max_pairwise_product(n, a):
                global result
                for i in range(0, n):
                    for j in range(i + 1, n):
                        if a[i] * a[j] &gt; result:
                            result = a[i] * a[j]
                return result


            def max_pairwise_product_fast(n, a):
                global fast
                max_index1 = -1
                for i in range(n):
                    if max_index1 == -1 or a[i] &gt; a[max_index1]:
                        max_index1 = i

                max_index2 = -1
                for i in range(n):
                    if i != max_index1 and (max_index2 == -1 or a[i] &gt; a[max_index2]):
                        max_index2 = i
                fast = a[max_index1] * a[max_index2]
                return fast

            print(fast, result, ""OK"")
    else:
        print(""Wrong Answer"")
</code></pre>
",https://stackoverflow.com/questions/48092288/basic-python-algorithm-stress-test,"['python', 'arrays', 'algorithm', 'random', 'stress-testing']",8155,TRUE,0,1
"JAVA Performance insights , refactoring existing code","<p>I have a code like below, objective is to convert output of data layer to a generic data format so it can be used by other layers. </p>

<p>During my LT runs, i am observing this method consuming a considerable percentage of cpu time but still comparing to over all time it looks manageable.</p>

<p>But worried when it goes for stress testing it might blow up, so thinking for refactoring to remove object mapper usage and convert it by iteration.</p>

<p>I am not really a fan of converting a object to string and then back to another object structure. My belief is to avoid conversions(object-->String-->Object) and it will save time and memory is this is correct assumption?</p>

<pre><code>ObjectMapper om = new com.fasterxml.jackson.databind.ObjectMapper();
String listGridData = om.writeValueAsString(com.fasterxml.jackson.databind.node.ArrayNode grid);
List&lt;Map&lt;String, Object&gt;&gt;   responseList = om.readValue(listGridData,
                    new TypeReference&lt;List&lt;Map&lt;String, Object&gt;&gt;&gt;() {
                    });
</code></pre>
",https://stackoverflow.com/questions/51864980/java-performance-insights-refactoring-existing-code,"['java', 'jackson']",169,TRUE,1,1
white box between scroll wheels,"<p>I am having a bit of a CSS styling problem with an analytics dashboard that I am setting up. I've got the styling right for the table and scrollbars, but I cannot seem to figure out how to get rid of the white square in the bottom right hand corner of my table. I know that I can get rid of the horizontal scrollbar and that should fix the issue, however I need the scroll overflow in case there are long strings.</p>

<p>Relevant CSS:</p>

<pre><code>.keen-dataviz-table {
  border-bottom: 0px!important;
}

.keen-dataviz-table::-webkit-scrollbar {
  width: 12px;
}

.keen-dataviz-table::-webkit-scrollbar-track {
  -webkit-box-shadow: inset 0 0 6px rgba(0, 0, 0, 0.3);
  border-radius: 10px;
}

.keen-dataviz-table::-webkit-scrollbar-thumb {
  border-radius: 10px;
  -webkit-box-shadow: inset 0 0 6px rgba(0, 0, 0, 0.5);
}
</code></pre>

<p>The chart HTML is generated by Keen.io, the only HTML I have on my page for this is the following:</p>

<pre><code>&lt;div class=""col-sm-4""&gt;
  &lt;div id=""chart-01""&gt;&lt;/div&gt;
  &lt;br&gt;
&lt;/div&gt;
</code></pre>

<p>I've attached an image below for reference (please ignore the silly intents, I was just stress-testing some NLP intents/entities and the horizontal overflow of this table).</p>

<p>Any help would be greatly appreciated!</p>

<p><a href=""https://i.stack.imgur.com/VEnPE.png"" rel=""nofollow noreferrer""><img src=""https://i.stack.imgur.com/VEnPE.png"" alt=""Example of CSS Styling issue""></a></p>
",https://stackoverflow.com/questions/48737824/white-box-between-scroll-wheels,"['jquery', 'css', 'html']",736,TRUE,-1,1
Should C++ file read be slower than Ruby or C#?,"<p>Completely new to C++. </p>

<p>I'm comparing various aspects of C++, C# and Ruby to see if there's need for mirroring a library. Currently, simple read of a file (post update).</p>

<p>Compiling C++ and C# in VS 2017. C++ is in release(x64) mode (or at least compile then run)</p>

<p>The libraries more or less read a file and split the lines into three which make up the members of an object which are then stored in an array member.</p>

<p>For stress testing I tried a large file 380MB(7M lines) (after update) now getting similar performance with C++ and Ruby,</p>

<p>Purely reading the file and doing nothing else the performance is as below:</p>

<pre><code>Ruby: 7s
C#:   2.5s
C++:  500+s (stopped running after awhile, something's clearly wrong)
C++(release build x64): 7.5s
</code></pre>

<p>The code:</p>

<pre><code>#Ruby
file = File.open ""test_file.txt""
while !file.eof 
    line = file.readline
end

//C#
StreamReader file = new StreamReader(""test_file.txt"");
file.Open();
while((line = file.ReadLine()) != null){

}



//C++
#include ""stdafx.h""
#include ""string""
#include ""iostream""
#include ""ctime""
#include ""fstream""
int main()
{
    std::ios::sync_with_stdio(false);
    std::ifstream file;
    file.open(""c:/sandboxCPP/test_file.txt"");
    std::string line;

    std::clock_t start;
    double duration;
    start = std::clock();
    while (std::getline(file, line)) {

    }
    duration = (std::clock() - start) / (double)CLOCKS_PER_SEC;
    std::cout &lt;&lt; ""\nDuration: "" &lt;&lt; duration;
    while (true) 
    {

    }
    return 0;
}
</code></pre>

<p>Edit: The following performed incredibly well. 0.03s</p>

<pre><code>vector&lt;string&gt; lines;
string tempString = str.str();
boost::split(lines, tempString, boost::is_any_of(""\n""));
start = clock();
cout &lt;&lt; ""\nCount: "" &lt;&lt; lines.size();
int count = lines.size();
string s;
for (int i = 0; i &lt; count; i++) {
    s =  lines[i];
} 
</code></pre>

<p><code>s =</code> on the likelihood that I don't know what boost is doing. Changed performance.</p>

<p>Tested with a cout of a random record at the end of the loop.</p>

<p>Thanks</p>
",https://stackoverflow.com/questions/48121374/should-c-file-read-be-slower-than-ruby-or-c,"['c++', 'ifstream']",202,TRUE,-7,2
How to make API call in JMeter after specific time has passed,"<p>I am having the following situation in JMeter script:</p>

<ol>
<li>User gets ACCESS Token and logs in</li>
<li>50 GET APIs in system.</li>
<li>50 Threads.</li>
<li>Loop is set to FOREVER.</li>
<li>Duration is set to 10800 seconds (3 hours)</li>
<li>Login method where User gets the token is placed inside <em>Once Only Controller</em> so that Login is executed just once per thread.</li>
<li>User times out in 60min and gets logged out automatically.</li>
<li>There exists a Refresh token method which gets a new ACCESS token.</li>
</ol>

<p>I need to perform stress testing using JMeter using 50 threads and run it for 3 hours. The thing is if 50 threads are running, they will start getting time out after 60min.</p>

<p>Now if I place the Refresh Token method to get a new ACCESS token after the 50 API calls, then is there a way where I call this method when it is close to 60 min of execution, for eg: Refresh token method is called after every 55min?</p>
",https://stackoverflow.com/questions/52180674/how-to-make-api-call-in-jmeter-after-specific-time-has-passed,"['time', 'jmeter']",1530,TRUE,0,2
BiometricPrompt and AccessibilityService,"<p>Previously, FingerprintManager could be used to handle fingerprint authentication, and did not depend on an Activity in order to go through a system dialog.
It could be used from anywhere, as any of the sensor managers.</p>

<p>Now, Google have released the support library version in <a href=""https://developer.android.com/reference/androidx/biometrics/BiometricPrompt"" rel=""nofollow noreferrer"">https://developer.android.com/reference/androidx/biometrics/BiometricPrompt</a>
and deprecated FingerprintManager.
They wish all apps to go through the same unified system dialog.</p>

<p>Big problem is - the whole thing is strongly coupled to FragmentActivity.</p>

<p>I wish to use this fingerprint authentication in an AccessibilityService (display over other apps).</p>

<p>What can be done?</p>

<p>Some options I'm considering:</p>

<ol>
<li>Go back to the old FingerprintManager despite becoming deprecated.</li>
<li>Implement some transparent Activity that will somehow pop up above other apps, just to satisfy the API's requirement for FragmentActivity.</li>
</ol>

<p><strong><em>UPDATE</em></strong></p>

<p>Accepted commonsware's answer below. 
I am able to succesfully launch an Activity on top of apps when getting accessibility events, and start using BiometricPrompt. BiometricPrompt has other problems though, and it seems like a very half-baked API that never went through stress testing - but that's beyond the scope of this question.</p>
",https://stackoverflow.com/questions/53574218/biometricprompt-and-accessibilityservice,"['android', 'accessibilityservice', 'android-fingerprint-api', 'accessibility', 'accessibility-api']",1025,TRUE,2,1
Go&#39;s Garbage collector isn&#39;t releasing memory at the right time,"<p>I wrote a simple http golang webserver to test the Go garbage collection (releasing memory of inaccessible pointers), But in strees testing ,I understand it consumes excessive ram.</p>

<p>According to some <a href=""https://stackoverflow.com/questions/24376817/go-1-3-garbage-collector-not-releasing-server-memory-back-to-system"">question/answers</a>, I found out Golang does garbage collection automatically, manage the extra memory itself and doesn't get back memory to OS at once. Some results:</p>

<ul>
<li><p>The webserver release ram after about 5mins.</p></li>
<li><p>The webserver consumes ram until it gets up to specific level and requesting more connections not gets more memory(about 4GB in my case)</p></li>
</ul>

<p>Consuming 4GB of ram isn't cool! So i added a Goroutine to my code that gets the extra memory back to OS. </p>

<p>The code:</p>

<pre><code>type t struct{
    a       []string
    b       map[string]string
}
var x t = t{
    a:  []string{""1"",""2""},
    b:  make(map[string]string),
}

func handler(w http.ResponseWriter, r *http.Request) {
    x := &amp;t{}
    fmt.Fprintf(w, ""pong"", r.URL.Path[1:])
}

func main() {
    http.HandleFunc(""/ping"", handler)
    go func(){
        for {
            time.Sleep(10 * time.Second)
            fmt.Println(""Free up memory..."")
            debug.FreeOSMemory() 
        }

    }()
    log.Fatal(http.ListenAndServe("":8080"", nil))
}
</code></pre>

<p>For stress testing i use <a href=""https://httpd.apache.org/docs/2.4/programs/ab.html"" rel=""nofollow noreferrer"">Apache Bench</a>:</p>

<pre><code>ab -c 100 -n 400000 http://127.0.0.1:8080/ping
</code></pre>

<p>I ran above command about 10 times continuously to check the ram usage. 
It works very well, But the problem is, Is there any standard way that Go releases the memory better!? </p>

<p>I know in some cases, it's good that Go manages the memory itself instead of repetitive releasing and getting back memory from OS, But in my case, 5 minutes delay is very much to releasing memory.</p>
",https://stackoverflow.com/questions/49963498/gos-garbage-collector-isnt-releasing-memory-at-the-right-time,"['go', 'garbage-collection']",5571,TRUE,1,1
CLR GC thread behavior: SafeFileHandle unexpectedly finalized,"<p>We recently hit some issues that may be related to the GC behavior of CLR.</p>

<p>The problem I encountered is as follows:</p>

<p>We have a long running stress testing application written in C# that keeps opening file handles on a remote SMB file share (which is Azure Files Service), and uses those handles to perform file system operations like read/write, etc. </p>

<p>Typically we’ll keep those handle open for quite a long time, as we’ll use them repeatedly. But sometimes when we try to access some of those opened handles, we found that these handles were closed already. And from the trace logs captured by Process Monitor (one sample below):</p>

<p><br>fltmgr.sys!FltpPerformPreCallbacks+0x324
<br>fltmgr.sys!FltpPassThroughInternal+0x8c
<br>fltmgr.sys!FltpPassThrough+0x169
<br>fltmgr.sys!FltpDispatch+0x9e
<br>ntoskrnl.exeIopCloseFile+0x146
<br>ntoskrnl.exeObpDecrementHandleCount+0x9a
<br>ntoskrnl.exeNtClose+0x3d9
<br>ntoskrnl.exeKiSystemServiceCopyEnd+0x13
<br>ntdll.dll!ZwClose+0xa
<br>KERNELBASE.dll!CloseHandle+0x17
<br>mscorlib.ni.dll!mscorlib.ni.dll!+0x566038
<br>clr.dll!CallDescrWorkerInternal+0x83
<br>clr.dll!CallDescrWorkerWithHandler+0x4a
<br>clr.dll!DispatchCallSimple+0x60
<br>clr.dll!SafeHandle::RunReleaseMethod+0x69
<br>clr.dll!SafeHandle::Release+0x152
<br>clr.dll!SafeHandle::Dispose+0x5a
<br>clr.dll!SafeHandle::DisposeNative+0x9b
<br>mscorlib.ni.dll!mscorlib.ni.dll!+0x48d9d1
<br>mscorlib.ni.dll!mscorlib.ni.dll!+0x504b83
<br>clr.dll!FastCallFinalizeWorker+0x6
<br>clr.dll!FastCallFinalize+0x55
<br>clr.dll!MethodTable::CallFinalizer+0xac
<br>clr.dll!WKS::CallFinalizer+0x61
<br>clr.dll!WKS::DoOneFinalization+0x92
<br>clr.dll!WKS::FinalizeAllObjects+0x8f
<br>clr.dll!WKS::FinalizeAllObjects_Wrapper+0x18
<br>clr.dll!ManagedThreadBase_DispatchInner+0x2d
<br>clr.dll!ManagedThreadBase_DispatchMiddle+0x6c
<br>clr.dll!ManagedThreadBase_DispatchOuter+0x75
<br>clr.dll!ManagedThreadBase_DispatchInCorrectAD+0x15
<br>clr.dll!Thread::DoADCallBack+0xff
<br>clr.dll!ManagedThreadBase_DispatchInner+0x1d822c
<br>clr.dll!WKS::DoOneFinalization+0x145
<br>clr.dll!WKS::FinalizeAllObjects+0x8f
<br>clr.dll!WKS::GCHeap::FinalizerThreadWorker+0xa1
<br>clr.dll!ManagedThreadBase_DispatchInner+0x2d
<br>clr.dll!ManagedThreadBase_DispatchMiddle+0x6c
<br>clr.dll!ManagedThreadBase_DispatchOuter+0x75
<br>clr.dll!WKS::GCHeap::FinalizerThreadStart+0xd7
<br>clr.dll!Thread::intermediateThreadProc+0x7d
<br>KERNEL32.dll!BaseThreadInitThunk+0x1a
<br>ntdll.dll!RtlUserThreadStart+0x1d</p>

<p>It seems that the handles were closed in CLR GC Finalizer thread. However, our handles are opened in the following pattern which should not be GC’ed:</p>

<p>We use P/Invoke to open a file handle and obtain a SafeFileHandle and use that SafeFileHandle to construct a FileStream, and we’ll save the FileStream object in another object defined as follows:</p>

<pre><code>public class ScteFileHandle
{
    /// &lt;summary&gt;
    /// local file handle
    /// &lt;/summary&gt;
    [NonSerialized]
    public FileStream FileStreamHandle;

    /*
     * Some other fields
     */
</code></pre>

<p>}</p>

<p>P/Invoke we use:</p>

<pre><code>    [DllImport(""kernel32.dll"", SetLastError = true, CharSet = CharSet.Auto)]
    public static extern SafeFileHandle CreateFile(
       string lpFileName,
       Win32FileAccess dwDesiredAccess,
       Win32FileShare dwShareMode,
       IntPtr lpSecurityAttributes,
       Win32FileMode dwCreationDisposition,
       Win32FileAttributes dwFlagsAndAttributes,
       IntPtr hTemplateFile); 
</code></pre>

<p>SafeFileHandle fileHandle = Win32FileIO.CreateFile(fullFilePath, win32FileAccess, win32FileShare, IntPtr.Zero, win32FileMode, win32FileAttr, IntPtr.Zero);</p>

<p>FileStream fileStream = new FileStream(fileHandle, fileAccess, Constants.XSMBFileSectorSize);</p>

<p>One thing we’re sure of is that during the whole lifetime of our stress testing application, we definitely keep a reference to the ScteFileHandle object, so it will never be cleaned up by GC. However, we do have observed the SafeHandle referenced within the ScteFileHandle ‘s FileStream got finalized in CLR GC thread, as pasted in above trace log.</p>

<p>So I’m wondering what caused the SafeFileHandle to be GC’ed and if there’s any approach to avoid this ? I’m not familiar with the CLR GC behavior but from my perspective, the SafeFileHandle is not supposed to be GC’ed.</p>

<p>Any pointer or insight is greatly appreciated ! Please let me know if any other detail you need to diagnostic this issue : ) </p>
",https://stackoverflow.com/questions/51631497/clr-gc-thread-behavior-safefilehandle-unexpectedly-finalized,"['c#', '.net', 'garbage-collection', 'safefilehandle']",188,FALSE,0,0
Node .on method firing too many times,"<p>I have an Electron app that presents a directory listing to the user. When the user clicks a button my interface script, interface.js, clears the container div and sends a message to main.js. On receiving the message, main.js scans the directory into an array of filenames and returns the array to interface.js as a response. Interface.js uses a .on method that fires when the response is received and updates the container div with the contents of the array.</p>

<p>This is my first real attempt at using Node, and as far as interface behaviour went everything worked brilliantly! Wonderful, it's only been a few hours and I'm already loving Node!</p>

<p>However, while debugging/stress testing I printed the returned array within the .on method to the console and noticed some strange behaviour. The first time the user clicks the button, the .on method runs once (verified by one message to the console). The second time the user clicks, the method runs twice (verified by two messages to the console); the third time it runs three times and so on.</p>

<p>The function in main.js that scans the directory only runs once per click, so the issue must be within inteface.js.</p>

<p>My code for main.js and interface.js:</p>

<p>main.js:</p>

<pre><code>const {app, BrowserWindow, ipcMain} = require('electron');
const fs = require('fs');

...

ipcMain.on( 'list-directory', ( event, directory ) =&gt; {
    var files = fs.readdirSync( directory );
    event.sender.send( 'list-directory-reply', files );
});
</code></pre>

<p>interface.js</p>

<pre><code>var { ipcRenderer, remote } = require( 'electron' );  
var main = remote.require( ""./main.js"" );

...

button.addEventListener('click', function(){ showDialogue( this ); }, false );

...

showDialogue( select ) {
    // clear the dialogue
    // some other stuff
    ipcRenderer.send( 'list-directory', './files/documents/' );
    ipcRenderer.on( 'list-directory-reply', function( event, contents ) {
        console.log( contents );
        if ( contents.length &gt; 0 ) {
            // add contents to the dialogue
        }
    } );
}
</code></pre>

<p>The code is adapted from a tutorial on the Electron website.</p>

<p>Why does <code>ipcRenderer.on</code> run multiple times? Is it possible that it's bound to something every time the button is clicked, and thus runs as many times as past clicks? I put a print statement inside the event listener function, and inside the <code>showDialogue</code> function before the <code>ipcRenderer</code> stuff, but they both only printed once per click so the repeats are definitely only coming from <code>ipcRenderer.on</code>. </p>
",https://stackoverflow.com/questions/52111151/node-on-method-firing-too-many-times,"['javascript', 'node.js', 'electron']",3284,TRUE,9,2
View with accumulated self referencing calculation in SQL,"<p>I'm not even quite sure what to call what I am trying to do.  I need a column that accumulates the total of its own calculations on previous rows.  Here's a simplified example of the output I need (d is the column I need help with):</p>

<pre><code>a   b   c   d   e
1   36  6   36  6
2   0   5   30  6
3   0   4   24  6
4   22  10  40  4
5   0   9   36  4
6   0   8   32  4
</code></pre>

<p>a is an autonumber column</p>

<p>b is user entered column</p>

<p>c is a column that usually counts down by one, but occasionally has other values added</p>

<p>d is equal to: the prior row of d + b - prior row of e</p>

<p>e is equal to d/c</p>

<p>It may help if I explain why I need this.  I'm working on simulating the impact of rate shocks on a banking institution (commonly called stress testing).  b is a column that represents some impact of an interest rate shock (example: a percentage change to the number of loans made).  c is a column that represents the number of months before the market is expected to return to normal without any further rate shocks.  d represents the impact on loans for that time period and e is the rate at which d is returning to zero.</p>

<p>Here is what I've tried so far and why it doesn't work:</p>

<pre><code>SET var := 0;

SELECT var := var + b - var/c AS d
</code></pre>

<p>This would be my preferred solution, but I'm creating a view, and views are not allowed to reference user created variables.  It has to be a view because I need to reference it in other queries and views.</p>

<pre><code>SELECT (lag(d,1,0) OVER (ORDER BY a)) + b - (lag(e,1,0) OVER (ORDER BY a)) AS d
</code></pre>

<p>This doesn't work because d hasn't been defined yet.  I just get an error that d isn't a column.</p>

<pre><code>SELECT
     ((SELECT sum(temp.b)
     FROM table as temp
     WHERE temp.a &lt;= a)
     /(SELECT sum(temp2.c)
     FROM table as temp2
     WHERE temp2.a = a))
</code></pre>

<p>This doesn't work because I need to divide the reduced value of d, not just the sum of b.</p>
",https://stackoverflow.com/questions/50278824/view-with-accumulated-self-referencing-calculation-in-sql,"['mysql', 'sql', 'variables', 'recursion']",133,FALSE,1,3
Microsoft Unity HierarchicalLifetimeManager Deadlock,"<p>I have a self hosted async c# webapi that is using OWIN and Unity 5.5.0. </p>

<p>I encountered an issue during some stress testing where after several hours the webapi would stop processing requests and the client calls would timeout. </p>

<p>I generated a crash dump using adplus in hang mode, opened the dump  in visual studio and found that my application was getting locked on the unity method SynchronizedLifetimeManager.GetValue()</p>

<p><a href=""https://i.stack.imgur.com/p8wGP.png"" rel=""nofollow noreferrer"">Threads Blocked by SynchronizedLifetimeManager.GetValue</a></p>

<p>the code from that method should be</p>

<pre><code>    /// &lt;summary&gt;
    /// Retrieve a value from the backing store associated with this Lifetime policy.
    /// &lt;/summary&gt;
    /// &lt;returns&gt;the object desired, or null if no such object is currently stored.&lt;/returns&gt;
    /// &lt;remarks&gt;Calls to this method acquire a lock which is released only if a non-null value
    /// has been set for the lifetime manager.&lt;/remarks&gt;
    public override object GetValue()
    {
        Monitor.Enter(lockObj);
        object result = SynchronizedGetValue();
        if (result != null)
        {
            Monitor.Exit(lockObj);
        }
        return result;
    }
</code></pre>

<p>My understanding of the method is a request comes in to retrieve the object, if the object has not been set it returns but leaves the lock. When the object gets set the lock the monitor is exited and threads should be able to call. </p>

<p>My visible problem is that at some point unity does not have a value for that lifetime manager and blocks the threads, but set is not called to release them.</p>

<p>Option 1: at some point the value in the lifetime manager was changed to null. What would cause this to happen?
Option 2: The SetValue call to release the lock is failing or not being made at all. What causes the value to get set?</p>

<p>My assumption is that somewhere in my code an exception is not properly handled and is causing this issue, but I am not sure where to start looking and was hoping for a better understanding of what unity is trying to do.</p>

<p>The application has a class where we register interfaces to types for IOC. This is used for most mappings. However, it is a workflow application so at one point in all calls we look up what workflow class to call and tell unity to resolve the type string and inject any objects needed.</p>
",https://stackoverflow.com/questions/51293827/microsoft-unity-hierarchicallifetimemanager-deadlock,"['c#', 'unity-container']",195,FALSE,4,0
Setting ImageViews without blocking the UI thread by caching Bitmaps,"<h1>Introduction</h1>

<p>This is going to be a long one. I spent most of my weekend trying different approaches and am putting this up to hopefully help others with similar issues. I don’t have high hopes to find a perfect solution that solves everything, but maybe someone here can surprise me. :-)</p>

<h1>Context</h1>

<p>I have 4 <code>ImageView</code> in my Android app representing playing cards. When new cards are dealt, an <code>Animation</code> (""<em>HideAnimation</em>"") shrinks the old cards. Then, 250ms later, <code>onAnimationEnd</code>, the image in the <code>ImageView</code> is replaced and another <code>Animation</code>  (""<em>ShowAnimation</em>"") enlarges it, which looks like the cards flipping around.<br>
The cards are loaded from .png files in the Android drawable folder.</p>

<p><img src=""https://i.stack.imgur.com/1zZbN.gif"" width=""100"" height=""141""><br>
This shows two animation styles. One flips through the deck, the other changes directly.</p>

<h1>Issue</h1>

<p>My first implementation used <code>setImageResource</code> directly in the UI thread:</p>

<pre><code>ImageView card = findViewById(R.id.cardA);
[…]
public void onAnimationEnd(Animation animation) {
    […]
    card.setImageResource(R.drawable.deck);
    […]
}
</code></pre>

<p>This worked, and never crashed, but you could clearly see the delay it caused to decode the resource, with a noticeable lag in the animation.<br>
What I needed, was to decode the resource beforehand, so I could quickly swap the image when it was time, but this is where the pain started, accompanied by the dreadful <code>OutOfMemoryError</code> (OOME).  </p>

<p>(Side note: Obviously, the first thing was to ensure the image sizes were reasonable, which they are: Their resolution is chosen according to their size on a Full HD device, at 507x720 per image. Because of this, downsampling the image, as per <a href=""https://developer.android.com/topic/performance/graphics/load-bitmap.html"" rel=""nofollow noreferrer"">Loading Large Bitmaps Efficiently</a> does not apply. Forcing a downsampling noticeably reduces image quality.)</p>

<h1>First attempt - LruCache</h1>

<p>The first solution I tried, was using the <code>LruCache</code> as described on the Android Developer site: <a href=""https://developer.android.com/topic/performance/graphics/cache-bitmap.html"" rel=""nofollow noreferrer"">Caching Bitmaps</a>. It uses a <code>LinkedHashMap</code> to keep recently referenced objects. However, this solution had a memory problem:  </p>

<ul>
<li><p>Variant 1: I limited the LruCache to 5 bitmaps, since I have to cache exactly 5 bitmaps (one for each card, plus the deck image). This worked fine first, but then caused an <code>OutOfMemoryError</code> during “stress testing” (running the animation in fast succession, e.g. spamming the “New cards” button).  </p></li>
<li><p>Variant 2: I used the suggested memory limit. However, limiting to 1/8 available memory was only enough to cache a single, maybe two bitmaps, so the lag was still present. Increasing this limit to 1/4 memory gave fluent animations, but occasionally caused OOME during “stress test”.</p></li>
</ul>

<p>(Side note: A possible work-around could be to significantly limit the rate at which new cards can be requested. While this would probably reduce the number of OOMEs, since it gives the Garbage Collector more time to run, it doesn’t feel like a fix, but rather a work-around, since the underlying problem, the bitmap allocation, COULD still cause an OOME in other cases.)</p>

<h1>Second attempt – Bitmap array</h1>

<p>My next attempt was much simpler. I simply have 5 bitmaps in an array.</p>

<pre><code>Bitmap[] cardCache = new Bitmap[5];
</code></pre>

<p>When the ""<em>HideAnimation</em>"" starts, I start decoding the bitmaps (<code>cn</code> being the card number 1-4, 0 for the deck side):</p>

<pre><code>public static void prepareBitmapInMemoryCache(final Context context, final int resourceId, final int cn) {
    Thread t = new Thread(new Runnable() {
        public void run() {
            final BitmapFactory.Options options = new BitmapFactory.Options();
            Bitmap drawable = BitmapFactory.decodeResource(context.getResources(), resourceId, options);
            if(drawable != null) {
                synchronized(cardCache) {
                    cardCache[cn] = drawable;
                }
            }
        }
    });
    t.start();
}
</code></pre>

<p>Then, when it is time to change the image and start the ""<em>ShowAnimation</em>"":</p>

<pre><code>card.setImageBitmap(getBitmapFromMemoryCache(this, resourceId, cn);
[...]
public static Bitmap getBitmapFromMemoryCache(Context context, int resourceId, int cn) {
    synchronized(cardCache) {
        if(cardCache[cn] != null &amp;&amp; !cardCache[cn].isRecycled()) {
            return cardCache[cn];
        }
    }
    return getBitmapFromResources(context, resourceId);
}
</code></pre>

<p>This also occasionally caused an <code>OutOfMemoryError</code> in <code>decodeResource</code> during “stress testing”.  </p>

<p>(Side note: I also tried caching <code>Drawable</code> instead of <code>Bitmap</code> with no change.)</p>

<p>So, I started getting creative. When getting the bitmap, I tried ensuring there was enough memory:</p>

<pre><code>private static Bitmap getBitmapFromResources(Context context, int resourceId) {
    int availableMemory = getAvailableMemoryInMB();
    final BitmapFactory.Options options = new BitmapFactory.Options();
    if(availableMemory &gt; 60) options.inSampleSize = 1;
    else options.inSampleSize = 2;
    return BitmapFactory.decodeResource(context.getResources(), resourceId, options);
}
private static int getAvailableMemoryInMB() {
    final Runtime runtime = Runtime.getRuntime();
    final long usedMemInMB = (runtime.totalMemory() - runtime.freeMemory()) / 1048576L;
    final long maxHeapSizeInMB = runtime.maxMemory() / 1048576L;
    final Long availHeapSizeInMB = maxHeapSizeInMB - usedMemInMB;
    return availHeapSizeInMB.intValue();
}
</code></pre>

<p>And when setting the bitmap, I tried actively calling <code>recycle</code> on the old bitmap:</p>

<pre><code>public static void setCardImage(ImageView card, Bitmap bitmap) {
    Drawable oldDrawable = card.getDrawable();
    Bitmap oldBitmap = null;
    if(oldDrawable != null &amp;&amp; oldDrawable instanceof BitmapDrawable) {
        oldBitmap = ((BitmapDrawable) oldDrawable).getBitmap();
    }
    if(drawable != null) card.setImageBitmap(bitmap);
    if(oldBitmap != null &amp;&amp; oldBitmap != cardCache[0]) oldBitmap.recycle();
}
</code></pre>

<p>But, alas, the <code>OutOfMemoryError</code> still occasionally returned (during “stress test”) and I am ready to give up and simply return to the <code>setImageResource</code> solution which I know is stable under all conditions.  </p>

<p>If you want to see for yourself, the app is on <a href=""https://play.google.com/store/apps/details?id=org.johis.game24"" rel=""nofollow noreferrer"">Google Play</a> and <a href=""https://www.amazon.com/gp/mas/dl/android?p=org.johis.game24"" rel=""nofollow noreferrer"">Amazon</a>. The current version 1.3.1 (16 March 2018) uses the “Bitmap array” solution described above, so you can try to provoke the OOME. Older versions use the <code>setImageResource</code> method, which has never crashed. I expect to issue an update tomorrow (19 Mar 2018) reverting to the old, stable method, if I can’t find a better solution.</p>
",https://stackoverflow.com/questions/49350969/setting-imageviews-without-blocking-the-ui-thread-by-caching-bitmaps,"['android', 'memory', 'bitmap', 'imageview', 'out-of-memory']",293,FALSE,2,0
boost::thread::join() crash trying to reference destroyed thread_info,"<p>Edit: I've created a self-contained app that causes the problem. It requires normally over a 1000 iterations of the loop (creation/run/join of threads), sometimes not hitting the crash until several thousand iterations:</p>

<pre><code>#include &lt;boost/thread.hpp&gt;
static void do_nothing() {}

int main() {
    int thread_count = 0;
    while (true) {
        thread_count++;
        boost::thread t1(boost::bind(&amp;do_nothing));
        if (t1.joinable()) {
            t1.join();
        }
    }
}
</code></pre>

<p>And here is the address sanitizer dump after it catches use-of-freed-memory:</p>

<pre><code>=================================================================
==96437==ERROR: AddressSanitizer: heap-use-after-free on address 0x058526f4 at pc 0x000a22cb bp 0xbffff4a8 sp 0xbffff4a4
WRITE of size 4 at 0x058526f4 thread T0
atos(96439,0x100357380) malloc: enabling scribbling to detect mods to free blocks
    #0 0xa22ca in boost::detail::atomic_decrement(int _Atomic*) sp_counted_base_clang.hpp:36
    #1 0xa21be in boost::detail::sp_counted_base::release() sp_counted_base_clang.hpp:115
    #2 0xa2157 in boost::detail::shared_count::~shared_count() shared_count.hpp:473
    #3 0xa115b in boost::detail::shared_count::~shared_count() shared_count.hpp:472
    #4 0x1ae8e63 in boost::thread::join_noexcept() shared_ptr.hpp:779
    #5 0x989a8c in boost::thread::join() thread.hpp:766
    #6 0x989366 in main main.cpp

0x058526f4 is located 4 bytes inside of 16-byte region [0x058526f0,0x05852700)
freed by thread T0 here:
    #0 0x35ca20d in wrap__ZdlPv (libclang_rt.asan_osx_dynamic.dylib:i386+0x6520d)
    #1 0x9ab89c in boost::detail::sp_counted_impl_p&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;::~sp_counted_impl_p() sp_counted_impl.hpp:53
    #2 0xa1dd1 in boost::detail::sp_counted_base::destroy() sp_counted_base_clang.hpp:97
    #3 0xa23e7 in boost::detail::sp_counted_base::weak_release() sp_counted_base_clang.hpp:131
    #4 0xa2262 in boost::detail::sp_counted_base::release() sp_counted_base_clang.hpp:118
    #5 0xa2157 in boost::detail::shared_count::~shared_count() shared_count.hpp:473
    #6 0xa115b in boost::detail::shared_count::~shared_count() shared_count.hpp:472
    #7 0x1ae8e56 in boost::thread::join_noexcept() shared_ptr.hpp:779
    #8 0x989a8c in boost::thread::join() thread.hpp:766
    #9 0x989366 in main main.cpp

previously allocated by thread T0 here:
    #0 0x35c9c0d in wrap__Znwm (libclang_rt.asan_osx_dynamic.dylib:i386+0x64c0d)
    #1 0x9ab357 in boost::detail::shared_count::shared_count&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_count.hpp:137
    #2 0x9ab214 in boost::detail::shared_count::shared_count&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_count.hpp:132
    #3 0x9aaf88 in void boost::detail::sp_pointer_construct&lt;boost::detail::thread_data_base, boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::shared_ptr&lt;boost::detail::thread_data_base&gt;*, boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*, boost::detail::shared_count&amp;) shared_ptr.hpp:284
    #4 0x9aae3b in boost::shared_ptr&lt;boost::detail::thread_data_base&gt;::shared_ptr&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_ptr.hpp:362
    #5 0x99c804 in boost::shared_ptr&lt;boost::detail::thread_data_base&gt;::shared_ptr&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_ptr.hpp:361
    #6 0x99c407 in boost::shared_ptr&lt;boost::detail::thread_data_base&gt; boost::thread::make_thread_info&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;is_same&lt;boost::decay&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::type, boost::thread&gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:229
    #7 0x99c120 in boost::thread::thread&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;boost::thread_detail::is_rv&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:299
    #8 0x989826 in boost::thread::thread&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;boost::thread_detail::is_rv&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:300
    #9 0x989366 in main main.cpp

SUMMARY: AddressSanitizer: heap-use-after-free sp_counted_base_clang.hpp:36 in boost::detail::atomic_decrement(int _Atomic*)
Shadow bytes around the buggy address:
  0x20b0a480: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a490: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a4a0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a4b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x20b0a4c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=&gt;0x20b0a4d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fd]fd
  0x20b0a4e0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a4f0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a500: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a510: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x20b0a520: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb

 =================================================================
 ==96437==ERROR: AddressSanitizer: heap-use-after-free on address 0x058526f4 at pc 0x000a22cb bp 0xbffff4a8 sp 0xbffff4a4
 WRITE of size 4 at 0x058526f4 thread T0
     #0 0xa22ca in boost::detail::atomic_decrement(int _Atomic*) sp_counted_base_clang.hpp:36
     #1 0xa21be in boost::detail::sp_counted_base::release() sp_counted_base_clang.hpp:115
     #2 0xa2157 in boost::detail::shared_count::~shared_count() shared_count.hpp:473
     #3 0xa115b in boost::detail::shared_count::~shared_count() shared_count.hpp:472
     #4 0x1ae8e63 in boost::thread::join_noexcept() shared_ptr.hpp:779
     #5 0x989a8c in boost::thread::join() thread.hpp:766
     #6 0x989366 in main main.cpp

 0x058526f4 is located 4 bytes inside of 16-byte region [0x058526f0,0x05852700)
 freed by thread T0 here:
     #0 0x35ca20d in wrap__ZdlPv (libclang_rt.asan_osx_dynamic.dylib:i386+0x6520d)
     #1 0x9ab89c in boost::detail::sp_counted_impl_p&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;::~sp_counted_impl_p() sp_counted_impl.hpp:53
     #2 0xa1dd1 in boost::detail::sp_counted_base::destroy() sp_counted_base_clang.hpp:97
     #3 0xa23e7 in boost::detail::sp_counted_base::weak_release() sp_counted_base_clang.hpp:131
     #4 0xa2262 in boost::detail::sp_counted_base::release() sp_counted_base_clang.hpp:118
     #5 0xa2157 in boost::detail::shared_count::~shared_count() shared_count.hpp:473
     #6 0xa115b in boost::detail::shared_count::~shared_count() shared_count.hpp:472
     #7 0x1ae8e56 in boost::thread::join_noexcept() shared_ptr.hpp:779
     #8 0x989a8c in boost::thread::join() thread.hpp:766
     #9 0x989366 in main main.cpp

 previously allocated by thread T0 here:
     #0 0x35c9c0d in wrap__Znwm (libclang_rt.asan_osx_dynamic.dylib:i386+0x64c0d)
     #1 0x9ab357 in boost::detail::shared_count::shared_count&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_count.hpp:137
     #2 0x9ab214 in boost::detail::shared_count::shared_count&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_count.hpp:132
     #3 0x9aaf88 in void boost::detail::sp_pointer_construct&lt;boost::detail::thread_data_base, boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::shared_ptr&lt;boost::detail::thread_data_base&gt;*, boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*, boost::detail::shared_count&amp;) shared_ptr.hpp:284
     #4 0x9aae3b in boost::shared_ptr&lt;boost::detail::thread_data_base&gt;::shared_ptr&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_ptr.hpp:362
     #5 0x99c804 in boost::shared_ptr&lt;boost::detail::thread_data_base&gt;::shared_ptr&lt;boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt; &gt;(boost::detail::thread_data&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;*) shared_ptr.hpp:361
     #6 0x99c407 in boost::shared_ptr&lt;boost::detail::thread_data_base&gt; boost::thread::make_thread_info&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;is_same&lt;boost::decay&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::type, boost::thread&gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:229
     #7 0x99c120 in boost::thread::thread&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;boost::thread_detail::is_rv&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:299
     #8 0x989826 in boost::thread::thread&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;(boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt;, boost::disable_if_c&lt;boost::thread_detail::is_rv&lt;boost::_bi::bind_t&lt;void, void (*)(), boost::_bi::list0&gt; &gt;::value, boost::thread::dummy*&gt;::type) thread.hpp:300
     #9 0x989366 in main main.cpp

 SUMMARY: AddressSanitizer: heap-use-after-free sp_counted_base_clang.hpp:36 in boost::detail::atomic_decrement(int _Atomic*)
 Shadow bytes around the buggy address:
   0x20b0a480: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a490: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a4a0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a4b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
   0x20b0a4c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
 =&gt;0x20b0a4d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fd]fd
   0x20b0a4e0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a4f0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a500: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a510: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
   0x20b0a520: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
 Shadow byte legend (one shadow byte represents 8 application bytes):
   Addressable:           00
   Partially addressable: 01 02 03 04 05 06 07
   Heap left redzone:       fa
   Freed heap region:       fd
   Stack left redzone:      f1
   Stack mid redzone:       f2
   Stack right redzone:     f3
   Stack after return:      f5
   Stack use after scope:   f8
   Global redzone:          f9
   Global init order:       f6
   Poisoned by user:        f7
   Container overflow:      fc
   Array cookie:            ac
   Intra object redzone:    bb
   ASan internal:           fe
   Left alloca redzone:     ca
   Right alloca redzone:    cb
</code></pre>

<p>Build Environment: Boost 1.63 built under macOS LLVM 9.0 with libc++ (host app built with same). Host is running a debug version, so no compiler optimization. Boost was probably built with optimization on, but I'll have to look into that.</p>

<p>Anyone have any clue as to where I should look? TBH, I'm not sure what join_noexcept() is doing there with the shared_ptr resetting exactly – why is the necessary? I don't think this is a Boost bug, but I'm at a loss on where to look. The host app might be stomping on something of course, but I've looked into that extensively and haven't found anything yet. The crashes consistently occur when the join() is happening.</p>

<p><em>Original post before I created the minimal case example above:</em> I'm seeing infrequent crashes when waiting for a communication thread to shutdown via join(). The crashes happen infrequently, but I can reproduce it with some stress testing after a few hours of opening/closing a communication thread ~3 times per second.</p>

<p>I captured the crash with debug memory options enabled (zombie blocks, etc.), and it is showing that the in:</p>

<pre><code>thread::join_noexcept()
{
            ...
            if(thread_info==local_thread_info)
            {
                thread_info.reset();
            }
            ...
}
</code></pre>

<p>the conditional is True, so the reset() is performed resulting in the thread_data being destroyed with a stack that looks like:</p>

<pre><code>detail::sp_counted_impl_p&lt;boost::detail::thread_data&lt;CThreadAdapter&gt; &gt;::~sp_counted_impl_p() at sp_counted_impl.hpp:53
detail::sp_counted_base::destroy() at sp_counted_base_clang.hpp:97
detail::sp_counted_base::weak_release() at sp_counted_base_clang.hpp:131
detail::sp_counted_base::release() at sp_counted_base_clang.hpp:118
detail::shared_count::~shared_count() at shared_count.hpp:473
detail::shared_count::~shared_count() at shared_count.hpp:472
shared_ptr&lt;boost::detail::thread_data_base&gt;::~shared_ptr() [inlined] at shared_ptr.hpp:779
shared_ptr&lt;boost::detail::thread_data_base&gt;::~shared_ptr() [inlined] at shared_ptr.hpp:779
shared_ptr&lt;boost::detail::thread_data_base&gt;::reset() [inlined] at shared_ptr.hpp:667
thread::join_noexcept() at thread.cpp:343
thread::join() at thread.hpp:766
</code></pre>

<p>(above is not the crash, just where the memory is deallocated that is later referenced)</p>

<p>The crash then occurs in the same join_noexcept() call after it is complete when it is destroying its locals. It appears it tries to access the freed thread_data (I think):</p>

<pre><code>detail::atomic_decrement(int _Atomic*) at sp_counted_base_clang.hpp:36
detail::sp_counted_base::release() at sp_counted_base_clang.hpp:115
detail::shared_count::~shared_count() at shared_count.hpp:473
detail::shared_count::~shared_count() at shared_count.hpp:472
shared_ptr&lt;boost::detail::thread_data_base&gt;::~shared_ptr() [inlined] at shared_ptr.hpp:779
shared_ptr&lt;boost::detail::thread_data_base&gt;::~shared_ptr() [inlined] at shared_ptr.hpp:779
thread::join_noexcept() at thread.cpp:351
thread::join() at thread.hpp:766
</code></pre>
",https://stackoverflow.com/questions/51438287/boostthreadjoin-crash-trying-to-reference-destroyed-thread-info,"['c++', 'boost', 'shared-ptr', 'boost-thread']",834,FALSE,1,2
Why is my Go server memory leaking?,"<p>I've written a simple TCP server.<br>
The problem is that when stress-testing it, it seems that the memory usage is increasing dramatically, and not decreasing when he test is done.
When the server is started, it takes ~700KB.<br>
During and after the stress-test, the memory usage jumps to ~7MB.<br>
Here's my code:</p>

<pre><code>package main

import (
    ""net""
    ""log""
    ""fmt""
    ""bufio""
)

func main() {
    ln, err := net.Listen(""tcp"", "":8888"")
    if err != nil {
        log.Fatal(err)
    }
    defer ln.Close()
    for {
        conn, err := ln.Accept()
        if err != nil {
            fmt.Println(err)
            continue
        }
        go handle(conn)
    }
}

func handle(conn net.Conn) {
    defer conn.Close()
    fmt.Println(""Accepted"", conn.LocalAddr())
    for {
        buf, err := bufio.NewReader(conn).ReadString('\n')
        if err != nil {
            break
        }
        msg := string(buf[:len(buf)-2])
        fmt.Println(""Received"", msg)
        conn.Write([]byte(""OK\n""))
    }
}
</code></pre>

<p>Any help is much appreciated.  </p>

<p>Note: I'm using <code>tcpkali</code> for loading it. This is the command line:</p>

<pre><code>tcpkali -em ""testing\r\n"" -c 100 -r 1000 -T 60  127.0.0.1:8888
</code></pre>

<p>EDIT: Following some comments below, I ran some tests and here are the results:</p>

<ol>
<li>Started the server and ran <code>tcpkali</code>.</li>
<li>After the first run, RSS was at 8516.</li>
<li>After a second run, RSS climbed to 8572.</li>
<li>Server is now idle. 5 minutes later, RSS climbed to 8588.</li>
<li>5 more minutes later, RSS climbed to 8608, and seems stable.</li>
<li>After 15 minutes of break, I ran <code>tcpkali</code> again, and RSS climbed to 8684.</li>
<li>A few minutes break, another <code>tcpkali</code> run, RSS climbs to 8696.</li>
<li>A few minutes break, another <code>tcpkali</code> run, RSS climbs to 8704.</li>
<li>A few minutes break, another <code>tcpkali</code> run, RSS climbs to 8712.</li>
</ol>

<p>Now, I don't know what you call this, but I call this a memory leak. Something is wrong here. No memory is freed, and RSS is keep climbing every time I run a test. Obviously, this thing cannot be deployed to production as it will eventually consume all available memory.<br>
I also tried calling <code>os.FreeOSMemory()</code> but nothing happens.  </p>

<p>My system is Go 1.9.4 on macOS 10.13.1. Is this environment related or am I missing something?</p>

<p><strong>LAST UPDATE:</strong><br>
Following @Steffen Ullrich answer and the tests that failed on my environment, I gave it a try on Ubuntu server, and the memory is freed after a few minutes of idle time.<br>
Seems like there's an issue with macOS.</p>
",https://stackoverflow.com/questions/49842546/why-is-my-go-server-memory-leaking,['go'],1202,TRUE,0,1
Temporary tables and constant statement recompilation,"<p>I am stress testing a system that is using a temporary table in dynamic SQL. The table is created early on in the transaction and is filled by several dynamic SQL statements in several stored procedures that are executed as part of the batch using statements of the form:</p>

<pre><code>INSERT #MyTable (...)
SELECT ...
</code></pre>

<p>where the <code>SELECT</code> statement is reasonably complicated in that it may contain <code>UNION ALL</code> and <code>UNPIVOT</code> statements and refer to several UDFs. All strings are executed using <code>sp_executesql</code> and Parameter Sniffing is enabled.</p>

<p>I have noticed that under load I am seeing a lot of RESOURCE_SEMAPHORE_QUERY_COMPILE waits where the query text being recompiled is present and <em>identical</em> in several waits at the same time and appears throughout the stress test which lasts about 5mins. The memory consumption on the server usually sits around 60% utilization and there is no limit on how much SQL Server can consume. The limiting factor appears to be CPU, which is constantly at >95% during the test.</p>

<p>I have profiled the server during the test to observe the SQL:StmtRecompile event which highlights the reason for the recompile is:</p>

<blockquote>
  <p>5 - Temp table changed</p>
</blockquote>

<p>but the temp table is the same every time and there are no DDL statements performed against the table once it has been created, apart from when it is dropped at the end of the batch.</p>

<p>So far, I have tried:</p>

<ul>
<li>Enabling the ""optimize for ad hoc workloads"" option</li>
<li><code>OPTION(KEEPFIXED PLAN)</code></li>
<li>Changing the dynamic statement to just the <code>SELECT</code> and then using <code>INSERT ... EXEC</code> so the temp table is not in the executed string</li>
</ul>

<p>All of these have made no difference and the waits persist.</p>

<p>Why would SQL think that is needs to recompile these identical queries each time they are executed and how can I get it to keep and reuse the cached plans it is creating?</p>

<p>Note: I cannot change the temp table to an In-Memory table because sometimes the stored procedures using this may have to query another database on the same instance.</p>

<p>This is using SQL Server 2016 SP1 CU7.</p>
",https://stackoverflow.com/questions/48596901/temporary-tables-and-constant-statement-recompilation,"['sql-server', 'sql-server-2016', 'stress-testing']",1175,TRUE,2,1
Pinning user space buffer for DMA from Linux kernel,"<p>I'm writing driver for devices that produce around 1GB of data per second. Because of that I decided to map user buffer allocated by application directly for DMA instead of copying through intermediate kernel buffer.</p>

<p>The code works, more or less. But during long-run stress testing I see kernel oops with ""bad page state"" initiated by unrelated applications (for instance <code>updatedb</code>), probably when kernel wants to swap some pages:</p>

<pre><code>[21743.515404] BUG: Bad page state in process PmStabilityTest  pfn:357518
[21743.521992] page:ffffdf844d5d4600 count:19792158 mapcount:0 mapping:          (null) index:0x12b011e012d0132
[21743.531829] flags: 0x119012c01220124(referenced|lru|slab|reclaim|uncached|idle)
[21743.539138] raw: 0119012c01220124 0000000000000000 012b011e012d0132 012e011e011e0111
[21743.546899] raw: 0000000000000000 012101300131011c 0000000000000000 012101240123012b
[21743.554638] page dumped because: page still charged to cgroup
[21743.560383] page-&gt;mem_cgroup:012101240123012b
[21743.564745] bad because of flags: 0x120(lru|slab)
[21743.569555] BUG: Bad page state in process PmStabilityTest  pfn:357519
[21743.576098] page:ffffdf844d5d4640 count:18219302 mapcount:18940179 mapping:          (null) index:0x0
[21743.585318] flags: 0x0()
[21743.587859] raw: 0000000000000000 0000000000000000 0000000000000000 0116012601210112
[21743.595599] raw: 0000000000000000 011301310127012f 0000000000000000 012f011d010d011a
[21743.603336] page dumped because: page still charged to cgroup
[21743.609108] page-&gt;mem_cgroup:012f011d010d011a
...
Entering kdb (current=0xffff8948189b2d00, pid 6387) on processor 6 Oops: (null)
due to oops @ 0xffffffff9c87f469
CPU: 6 PID: 6387 Comm: updatedb.mlocat Tainted: G    B      OE   4.10.0-42-generic #46~16.04.1-Ubuntu
...
</code></pre>

<hr>

<p>Details:</p>

<p>The user buffer consists of frames and neither the buffer not the frames are page-aligned. The frames in buffer are used in circular manner for ""infinite"" live data transfers. For each frame I get memory pages via <code>get_user_pages_fast</code>, then convert it to scatter-gatter table with <code>sg_alloc_table_from_pages</code> and finally map for DMA using <code>dma_map_sg</code>.</p>

<p>I rely on <code>sg_alloc_table_from_pages</code> to bind consecutive pages into one DMA descriptor to reduce size of S/G table sent to device. Devices are custom built and utilize FPGA. I took inspiration from many drivers doing similar mapping, especially video drivers i915 and radeon, but no one has all the stuff on one place so I might overlook something.</p>

<p>Related functions (<code>pin_user_buffer</code> and <code>unpin_user_buffer</code> are called upon separate IOCTLs):</p>

<pre><code>static int pin_user_frame(struct my_dev *cam, struct udma_frame *frame)
{
        const unsigned long bytes = cam-&gt;acq_frame_bytes;
        const unsigned long first =
                ( frame-&gt;uaddr              &amp;  PAGE_MASK) &gt;&gt; PAGE_SHIFT;
        const unsigned long last =
                ((frame-&gt;uaddr + bytes - 1) &amp;  PAGE_MASK) &gt;&gt; PAGE_SHIFT;
        const unsigned long offset =
                  frame-&gt;uaddr              &amp; ~PAGE_MASK;
        int nr_pages = last - first + 1;
        int err;
        int n;
        struct page **pages;
        struct sg_table *sgt;

        if (frame-&gt;uaddr + bytes &lt; frame-&gt;uaddr) {
                pr_err(""%s: attempted user buffer overflow!\n"", __func__);
                return -EINVAL;
        }

        if (bytes == 0) {
                pr_err(""%s: user buffer has zero bytes\n"", __func__);
                return -EINVAL;
        }

        pages = kcalloc(nr_pages, sizeof(*pages), GFP_KERNEL | __GFP_ZERO);
        if (!pages) {
                pr_err(""%s: can't allocate udma_frame.pages\n"", __func__);
                return -ENOMEM;
        }

        sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
        if (!sgt) {
                pr_err(""%s: can't allocate udma_frame.sgt\n"", __func__);
                err = -ENOMEM;
                goto err_alloc_sgt;
        }

        /* (rw == READ) means read from device, write into memory area */
        err = get_user_pages_fast(frame-&gt;uaddr, nr_pages, READ == READ, pages);
        if (err &lt; nr_pages) {
                nr_pages = err;
                if (err &gt; 0) {
                        pr_err(""%s: can't pin all %d user pages, got %d\n"",
                               __func__, nr_pages, err);
                        err = -EFAULT;
                } else {
                        pr_err(""%s: can't pin user pages\n"", __func__);
                }
                goto err_get_pages;
        }

        for (n = 0; n &lt; nr_pages; ++n)
                flush_dcache_page(pages[n]); //&lt;--- Is this needed?

        err = sg_alloc_table_from_pages(sgt, pages, nr_pages, offset, bytes,
                                        GFP_KERNEL);
        if (err) {
                pr_err(""%s: can't build sg_table for %d pages\n"",
                       __func__, nr_pages);
                goto err_alloc_sgt2;
        }

        if (!dma_map_sg(&amp;cam-&gt;pci_dev-&gt;dev, sgt-&gt;sgl, sgt-&gt;nents, DMA_FROM_DEVICE)) {
                pr_err(""%s: can't map %u sg_table entries for DMA\n"",
                       __func__, sgt-&gt;nents);
                err = -ENOMEM;
                goto err_dma_map;
        }

        frame-&gt;pages = pages;
        frame-&gt;nr_pages = nr_pages;
        frame-&gt;sgt = sgt;

        return 0;

err_dma_map:
        sg_free_table(sgt);

err_alloc_sgt2:
err_get_pages:
        for (n = 0; n &lt; nr_pages; ++n)
                put_page(pages[n]);
        kfree(sgt);

err_alloc_sgt:
        kfree(pages);

        return err;
}

static void unpin_user_frame(struct my_dev *cam, struct udma_frame *frame)
{
        int n;

        dma_unmap_sg(&amp;cam-&gt;pci_dev-&gt;dev, frame-&gt;sgt-&gt;sgl, frame-&gt;sgt-&gt;nents,
                     DMA_FROM_DEVICE);

        sg_free_table(frame-&gt;sgt);
        kfree(frame-&gt;sgt);
        frame-&gt;sgt = NULL;

        for (n = 0; n &lt; frame-&gt;nr_pages; ++n) {
                struct page *page = frame-&gt;pages[n];
                set_page_dirty_lock(page);
                mark_page_accessed(page); //&lt;--- Without this the Oops are more frequent
                put_page(page);
        }
        kfree(frame-&gt;pages);
        frame-&gt;pages = NULL;

        frame-&gt;nr_pages = 0;
}

static void unpin_user_buffer(struct my_dev *cam)
{
        if (cam-&gt;udma_frames) {
                int n;
                for (n = 0; n &lt; cam-&gt;udma_frame_count; ++n)
                        unpin_user_frame(cam, &amp;cam-&gt;udma_frames[n]);
                kfree(cam-&gt;udma_frames);
                cam-&gt;udma_frames = NULL;
        }
        cam-&gt;udma_frame_count = 0;
        cam-&gt;udma_buffer_bytes = 0;
        cam-&gt;udma_buffer = NULL;
        cam-&gt;udma_desc_count = 0;
}

static int pin_user_buffer(struct my_dev *cam)
{
        int err;
        int n;
        const u32 acq_frame_count = cam-&gt;acq_buffer_bytes / cam-&gt;acq_frame_bytes;
        struct udma_frame *udma_frames;
        u32 udma_desc_count = 0;

        if (!cam-&gt;acq_buffer) {
                pr_err(""%s: user buffer is NULL!\n"", __func__);
                return -EFAULT;
        }

        if (cam-&gt;udma_buffer == cam-&gt;acq_buffer
            &amp;&amp; cam-&gt;udma_buffer_bytes == cam-&gt;acq_buffer_bytes
            &amp;&amp; cam-&gt;udma_frame_count == acq_frame_count)
                return 0;

        if (cam-&gt;udma_buffer)
                unpin_user_buffer(cam);

        udma_frames = kcalloc(acq_frame_count, sizeof(*udma_frames),
                              GFP_KERNEL | __GFP_ZERO);
        if (!udma_frames) {
                pr_err(""%s: can't allocate udma_frame array for %u frames\n"",
                       __func__, acq_frame_count);
                return -ENOMEM;
        }

        for (n = 0; n &lt; acq_frame_count; ++n) {
                struct udma_frame *frame = &amp;udma_frames[n];
                frame-&gt;uaddr =
                        (unsigned long)(cam-&gt;acq_buffer + n * cam-&gt;acq_frame_bytes);
                err = pin_user_frame(cam, frame);
                if (err) {
                        pr_err(""%s: can't pin frame %d (out of %u)\n"",
                               __func__, n + 1, acq_frame_count);
                        for (--n; n &gt;= 0; --n)
                                unpin_user_frame(cam, frame);
                        kfree(udma_frames);
                        return err;
                }
                udma_desc_count += frame-&gt;sgt-&gt;nents; /* Cannot overflow */
        }
        pr_debug(""%s: total udma_desc_count=%u\n"", __func__, udma_desc_count);

        cam-&gt;udma_buffer = cam-&gt;acq_buffer;
        cam-&gt;udma_buffer_bytes = cam-&gt;acq_buffer_bytes;
        cam-&gt;udma_frame_count = acq_frame_count;
        cam-&gt;udma_frames = udma_frames;
        cam-&gt;udma_desc_count = udma_desc_count;

        return 0;
}
</code></pre>

<p>Related structures:</p>

<pre><code>struct udma_frame {
        unsigned long   uaddr;      /* User address of the frame */
        int             nr_pages;   /* Nr. of pages covering the frame */
        struct page     **pages;    /* Actual pages covering the frame */
        struct sg_table *sgt;       /* S/G table describing the frame */
};

struct my_dev {
        ...
        u8 __user   *acq_buffer;   /* User-space buffer received via IOCTL */
        ...
        u8 __user   *udma_buffer;       /* User-space buffer for image */
        u32         udma_buffer_bytes;  /* Total image size in bytes */
        u32         udma_frame_count;   /* Nr. of items in udma_frames */
        struct udma_frame
                    *udma_frames;       /* DMA descriptors per frame */
        u32         udma_desc_count;    /* Total nr. of DMA descriptors */
        ...
};
</code></pre>

<p>Questions:</p>

<ol>
<li>How to properly pin user buffer pages and mark them as not movable?</li>
<li>If one frame ends and next frame starts in the same page, is it correct to handle it as two independent pages, i.e. pin the page twice?</li>
<li>The data comes from device to user buffer and app is supposed to not write to its buffer, but I have no control over it. Can I use <code>DMA_FROM_DEVICE</code> or rather 
use <code>DMA_BIDIRECTIONAL</code> just in case?</li>
<li>Do I need to use something like <code>SetPageReserved</code>/<code>ClearPageReserved</code> or <code>mark_page_reserved</code>/<code>free_reserved_page</code>?</li>
<li>Is IOMMU/swiotlb somehow involved? E.g. i915 driver doesn't use <code>sg_alloc_table_from_pages</code> if swiotlb is active?</li>
<li>What the difference between <code>set_page_dirty</code>, <code>set_page_dirty_lock</code> and <code>SetPageDirty</code> functions?</li>
</ol>

<p>Thanks for any hint.</p>

<p>PS: I cannot change the way the application gets the data without breaking our library API maintained for many years. So please do not advise e.g. to <code>mmap</code> kernel buffer...</p>
",https://stackoverflow.com/questions/49366040/pinning-user-space-buffer-for-dma-from-linux-kernel,"['c', 'linux-kernel', 'dma', 'pci-e', 'pinning']",2206,FALSE,1,1
Solve: Lock-Free Queue Multiple-Producer Multiple-Consumer - Memory Corruption,"<p>The LockFreeQueueMPMC <em>should</em> solve the MPMC problem without locks, but during the runtime there is memory corruption. LockFreeDispatchStackMPMC <em>does</em> solve the MPMC problem without locks and is used as a basis for the LockFreeCacheMPMC allocator. Both of these implementations pass stress testing. </p>

<p>The LockFreeQueueMPMC Enqueue performs the same operation as Lock FreeDispatchStackMPMC Send. This adds a new node to the list. The Dequeue operation is more complicated. Only one pointer can ever be cmpexg at a time so there is no solution with a Tail pointer. Inorder to remove a node from the list, the list needs to be walked and the last node removed. This changes the time for Dequeue from O(1) to O(N), but lock-free.  </p>

<p>The LockFreeDispatchStackMPMC is an out-of-order MPMC lock-free solution. Messages that arrived earliest get serviced first. This uses a stack instead of a queue, for some problems this is not acceptable as the messages must be ordered. This shows a 40%+ performance improvement over the Queue, if your messages can be unordered. </p>

<pre><code>template&lt;class T&gt;
struct Node
{
    std::atomic&lt;int&gt; Next;
    T *Data;
};
template&lt;class T&gt;
class LockFreeDispatchStackMPMC
{
public:
    LockFreeDispatchStackMPMC()
    {
        Head = NULL;
    }
    ~LockFreeDispatchStackMPMC(){
    }
    void Send(T *item)
    {
        Node&lt;T&gt; * new_node = Cache.Malloc();
        new_node-&gt;Data=item;
        bool done = false;
        while(!done)
        {
            auto head = Head.load();
            new_node-&gt;Next.store( head);
            if( Head.compare_exchange_weak(head,new_node))
            {
                done = true;
            }
        }
    }

    T *Recieve()
    {
        T *returnValue = NULL;
        bool done = false;
        while(!done)
        {
            auto head = Head.load();
            if(head == NULL)
            {
                done=true;
            }
            else
            {
                Node&lt;T&gt; * curr = head;
                Node&lt;T&gt; *next = curr-&gt;Next.load();
                if(Head.compare_exchange_weak(head,next))
                {
                    done = true;
                    returnValue = curr-&gt;Data;
                    curr-&gt;Next =NULL;
                    Cache.Free(curr);
                }

            }
        }
        return returnValue;
    }
public:
    std::atomic&lt;Node&lt;T&gt; *&gt; Head;

private:
    LockFreeMemCache&lt;Node&lt;T&gt; &gt; Cache;
};
</code></pre>

<p>This is a Cache of Objects container based on using two lists, usable as a cross-thread pool to store objects. This also allows reads from the graveyard as the objects have not been destroyed, writes should not be allowed. This is a problem in the queue algorithm. This is MPMC safe as well, even if every Node has to be allocated one at a time.</p>

<pre><code>#define GROW_BY_SIZE 4

template&lt;class T&gt;
class LockFreeCacheMPMC
{
public:
    LockFreeCacheMPMC()
    {
        Head=NULL;
        FreeStack=NULL;
        AddSomeCache();
    }
    ~LockFreeCacheMPMC()
    {
        Node&lt;T&gt; *node ,*prev;

        bool done = false;
        node = Head;
        prev = NULL;
        while(!done)
        {
            prev = node;
            if(node == NULL)
            {
                done = true;
            }
            else
            {
                node = node-&gt;Next.load();
                delete prev-&gt;Data;
                delete prev;
            }
        }
        done = false;
        node = FreeStack;
        prev = NULL;
        while(!done)
        {
            prev = node;
            if(node == NULL)
            {
                done = true;
            }
            else
            {
                node = node-&gt;Next.load();
                delete prev;
            }
        }
    }
    T *Malloc()
    {
        T *returnValue = NULL;
        returnValue=Pop();
        while(returnValue==NULL)
        {
            AddSomeCache();
            returnValue=Pop();
        }
        return returnValue;
    }
    void Free(T *ptr)
    {
        Push(ptr);
    }

private:
    void AddSomeCache()
    {

        for(int i=0; i &lt; GROW_BY_SIZE; i++)
        {
            T *tmp = new T();
            Push(tmp);
        }
    }

private:
    void Push(T *item)
    {
        Node&lt;T&gt; * new_node = PopNode(true);
        new_node-&gt;Data=item;

        bool done = false;
        while(!done)
        {
            Node&lt;T&gt;* head = Head.load();
            new_node-&gt;Next.store(head);
            if(Head.compare_exchange_weak(head,new_node))
            {
                done = true;
            }
        }
    }
    T *Pop()
    {
        T *returnValue = NULL;
        bool done = false;
        while(!done)
        {
            Node&lt;T&gt; * curr= Head.load();
            if(curr == NULL)
            {
                done=true;
            }
            else
            {
                Node&lt;T&gt; *next = curr-&gt;Next.load();
                if(Head.compare_exchange_weak(curr,next))
                {
                    done = true;
                    returnValue = curr-&gt;Data;
                    PushNode(curr);
                }
            }
        }
        return returnValue;
    }

    void PushNode(Node&lt;T&gt; *item)
    {
        item-&gt;Next = NULL;
        item-&gt;Data = NULL;
        bool done = false;
        while(!done)
        {
            Node&lt;T&gt;* fs = FreeStack.load();
            item-&gt;Next.store(fs);
            if(FreeStack.compare_exchange_weak(fs,item))
            {
                done = true;
            }
        }
    }
    Node&lt;T&gt; *PopNode(bool Alloc)
    {
        Node&lt;T&gt; *returnValue = NULL;

        bool done = false;
        while(!done)
        {
            Node&lt;T&gt; *fs = FreeStack.load();
            if(fs == NULL)
            {
                done=true;
            }
            else
            {
                Node&lt;T&gt; *next = fs-&gt;Next.load();
                if(FreeStack.compare_exchange_weak(fs,next))
                {
                    done = true;
                    returnValue = fs;
                }

            }
        }
        if ((returnValue == NULL) &amp;&amp;Alloc )
        {
            returnValue =new Node&lt;T&gt;();
            returnValue-&gt;Data = NULL;
            returnValue-&gt;Next = NULL;
        }
        return returnValue;
    }
    std::atomic&lt;Node&lt;T&gt; *&gt; Head;
    std::atomic&lt;Node&lt;T&gt; *&gt;FreeStack;
};
</code></pre>

<p>This is the problem class. It will run for awhile, but corruption happens. The problem is in the Dequeue method. Nodes get trimmed off the list one at a time. Its possible for a node to get trimmed out from underneath you at every step. This results in nodes that have been trimmed off and need to be 'deleted', but there are still active threads reading from the node. The algorithm should prevent any writes to a dead node as atomically a next pointer is either pointing at a node or null, but using a Cache Pool to store the nodes allows for safe reads from the graveyard. </p>

<pre><code>template&lt;class T&gt;
class LockFreeQueueMPMC
{
public:
    LockFreeQueueMPMC()
    {
        Head=NULL;
    }
    ~LockFreeQueueMPMC(){
    }
    void Enqueue(T *item)
    {
        Node&lt;T&gt; * new_node = Cache.Malloc();
        new_node-&gt;Data=item;

        bool done = false;
        while(!done)
        {
            auto head = Head.load();
            new_node-&gt;Next.store(head);
            if(Head.compare_exchange_weak(head,new_node))
            {
                done = true;
            }
        }
    }
    T *Dequeue()
    {
        T *returnValue=NULL;
        bool done = false;
        while(!done)
        {
            Node&lt;T&gt; *head = Head.load();
            if(head == NULL)
            {
                done = true;
            }
            else
            {
                Node&lt;T&gt; * prev, *curr;
                prev = NULL;
                curr = head;
                bool found = false;
                while(!found)
                {
                    if(curr == NULL)
                    {
                        break;
                    }
                    Node&lt;T&gt; * next = curr-&gt;Next.load();
                    if(next == NULL)
                    {
                        found=true;
                        break;
                    }
                    prev = curr;
                    curr = next;
                }
                if(found)
                {
                    if(prev == NULL)
                    {
                        if(Head.compare_exchange_weak(head,NULL))
                        {
                            done = true;
                        }
                    }
                    else
                    {
                        if(prev-&gt;Next.compare_exchange_weak(curr,NULL))
                        {
                            done = true;
                        }
                    }
                    if(done)
                    {
                        returnValue = curr-&gt;Data;
                        Cache.Free(curr);
                    }
                }
            }
        }
        return returnValue;
    }
private:
    std::atomic&lt;Node&lt;T&gt; *&gt; Head;
    LockFreeMemCache&lt;Node&lt;T&gt; &gt; Cache;
};
</code></pre>

<p>The problem is located in the Dequeue method, there is a step that corrupts, but only rarely. </p>
",https://stackoverflow.com/questions/53789955/solve-lock-free-queue-multiple-producer-multiple-consumer-memory-corruption,"['c++11', 'stack', 'queue', 'producer-consumer', 'memory-corruption']",114,FALSE,0,1
Foreground service START_STICKY is not restarted after being killed in low memory conditions,"<p>I know there are tones of threads with similar title, but no one of them gives a definitive answer. 
We're currently stress testing the foreground service of our application by filling up the RAM with junk with another app and investigating what happens to our application and its foreground service. </p>

<p>So, our foreground service is started and notification is shown to the user, and its mode is <code>START_STICKY</code>. However, when we simulate extreme memory usage the application is killed (which is OK and we can do nothing about it), but the bad thing is that our foreground service never gets restarted. Perhaps I'm confusing something from the START_STICKY doc, but what I can understand from below doc is that system should guarantee the restart, isn't it? </p>

<blockquote>
  <p>Constant to return from <code>onStartCommand(Intent, int, int)</code>: if this
  service's process is killed while it is started (after returning from
  <code>onStartCommand(Intent, int, int)</code>), then leave it in the started state
  but don't retain this delivered intent. <strong>Later the system will try to
  re-create the service</strong>. Because it is in the started state, it will
  guarantee to call <code>onStartCommand(Intent, int, int)</code> after creating the
  new service instance; if there are not any pending start commands to
  be delivered to the service, it will be called with a null intent
  object, so you must take care to check for this.</p>
</blockquote>

<p>The above doc is too vague. </p>

<p>First question is </p>

<ul>
<li>will it <strong>ALWAYS RESTART</strong> the service? If not, what are the
conditions for the service being restarted and what are the
conditions when it won't be restarted?</li>
</ul>

<p>And the second question is </p>

<ul>
<li>What does it mean <strong>Later</strong>? Later when? Later next year or what? :D</li>
</ul>

<p>To summarize, we do need to guarantee that our foreground service will restart whatever it takes. Any ideas? </p>

<hr>

<p><strong>EDIT 1:</strong>
I did investigate the logcat logs a little bit more. Here's is what happens.</p>

<pre><code>Process com.MY_PACKAGE_ID (pid 1192) has died: prcp FGS 
Scheduling restart of crashed service com.MY_PACKAGE_ID/MyForegroundService in 1284608ms
</code></pre>

<p>Obviously, the system kills the application and schedules a restart. In this case it's approximately in 21 minutes. In other cases I would see that the reschedule time was 10.000ms aka 10 seconds. And, in 10 seconds the device would still be in memory stressful situation and the app will not be launched. Which implies, that in case the service fails to start at rescheduled time there won't be any 2nd rescheduling and the service will never restart. But, if at rescheduled time the device was in normal non-memory stressful conditions, the restart will happen successfully. </p>
",https://stackoverflow.com/questions/53261349/foreground-service-start-sticky-is-not-restarted-after-being-killed-in-low-memor,"['android', 'service']",797,FALSE,3,0
