Login - Sorry, something went wrong Error 500 (Round 2)

This is very similar to this post and this post, but I don’t think it applies completely.

I am getting the following in my server pod logs:

{
    "component": "server",
    "severity": "INFO",
    "time": "2020-10-07T18:46:29.815Z",
    "environment": "production",
    "region": "local",
    "message": "(GitHub Auth) Callback (login)",
    "payload": {
        "login-flow": true,
        "clientInfo": {
            "ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15",
            "fingerprint": "b86c2eb8af26fab41d8c8c51f6d802f9e69216b7f1764a1a4197985c62bbb558"
        }
    }
}

{
    "@type": "type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent",
    "serviceContext": { "service": "server", "version": "v0.4.0" },
    "stack_trace": "Error at new RepositoryNotFoundError (/app/node_modules/typeorm/error/RepositoryNotFoundError.js:24:23) at EntityManager.getRepository (/app/node_modules/typeorm/entity-manager/EntityManager.js:753:19) at EmailDomainFilterDBImpl.<anonymous> (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:69:62) at step (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:42:23) at Object.next (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:23:53) at fulfilled (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:14:58) at propagateAslWrapper (/app/node_modules/async-listener/index.js:504:23) at /app/node_modules/async-listener/glue.js:188:31 at /app/node_modules/async-listener/index.js:541:70 at /app/node_modules/async-listener/glue.js:188:31 at process._tickCallback (internal/process/next_tick.js:68:7)",
    "component": "server",
    "severity": "ERROR",
    "time": "2020-10-07T18:46:29.814Z",
    "environment": "production",
    "region": "local",
    "message": "(GitHub Auth) Error in Verify Authentication Request",
    "error": "Error at new RepositoryNotFoundError (/app/node_modules/typeorm/error/RepositoryNotFoundError.js:24:23) at EntityManager.getRepository (/app/node_modules/typeorm/entity-manager/EntityManager.js:753:19) at EmailDomainFilterDBImpl.<anonymous> (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:69:62) at step (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:42:23) at Object.next (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:23:53) at fulfilled (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:14:58) at propagateAslWrapper (/app/node_modules/async-listener/index.js:504:23) at /app/node_modules/async-listener/glue.js:188:31 at /app/node_modules/async-listener/index.js:541:70 at /app/node_modules/async-listener/glue.js:188:31 at process._tickCallback (internal/process/next_tick.js:68:7)",
    "payload": {
        "login-flow": true,
        "err": "Error at new RepositoryNotFoundError (/app/node_modules/typeorm/error/RepositoryNotFoundError.js:24:23) at EntityManager.getRepository (/app/node_modules/typeorm/entity-manager/EntityManager.js:753:19) at EmailDomainFilterDBImpl.<anonymous> (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:69:62) at step (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:42:23) at Object.next (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:23:53) at fulfilled (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:14:58) at propagateAslWrapper (/app/node_modules/async-listener/index.js:504:23) at /app/node_modules/async-listener/glue.js:188:31 at /app/node_modules/async-listener/index.js:541:70 at /app/node_modules/async-listener/glue.js:188:31 at process._tickCallback (internal/process/next_tick.js:68:7)",
        "clientInfo": {
            "ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15",
            "fingerprint": "b86c2eb8af26fab41d8c8c51f6d802f9e69216b7f1764a1a4197985c62bbb558"
        }
    }
}

{
    "@type": "type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent",
    "serviceContext": { "service": "server", "version": "v0.4.0" },
    "stack_trace": "Error at new RepositoryNotFoundError (/app/node_modules/typeorm/error/RepositoryNotFoundError.js:24:23) at EntityManager.getRepository (/app/node_modules/typeorm/entity-manager/EntityManager.js:753:19) at EmailDomainFilterDBImpl.<anonymous> (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:69:62) at step (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:42:23) at Object.next (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:23:53) at fulfilled (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:14:58) at propagateAslWrapper (/app/node_modules/async-listener/index.js:504:23) at /app/node_modules/async-listener/glue.js:188:31 at /app/node_modules/async-listener/index.js:541:70 at /app/node_modules/async-listener/glue.js:188:31 at process._tickCallback (internal/process/next_tick.js:68:7)",
    "component": "server",
    "severity": "ERROR",
    "time": "2020-10-07T18:46:29.815Z",
    "environment": "production",
    "region": "local",
    "message": "(GitHub Auth) Redirect to /sorry",
    "error": "Error at new RepositoryNotFoundError (/app/node_modules/typeorm/error/RepositoryNotFoundError.js:24:23) at EntityManager.getRepository (/app/node_modules/typeorm/entity-manager/EntityManager.js:753:19) at EmailDomainFilterDBImpl.<anonymous> (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:69:62) at step (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:42:23) at Object.next (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:23:53) at fulfilled (/app/node_modules/@typefox/gitpod-db/lib/typeorm/email-domain-filter-db-impl.js:14:58) at propagateAslWrapper (/app/node_modules/async-listener/index.js:504:23) at /app/node_modules/async-listener/glue.js:188:31 at /app/node_modules/async-listener/index.js:541:70 at /app/node_modules/async-listener/glue.js:188:31 at process._tickCallback (internal/process/next_tick.js:68:7)",
    "payload": "{ request: IncomingMessage { _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: BufferList { head: null, tail: null, length: 0 }, length: 0, pipes: null, pipesCount: 0, flowing: null, ended: true, endEmitted: false, reading: false, sync: true, needReadable: false, emittedReadable: true, readableListening: false, resumeScheduled: false, paused: true, emitClose: true, destroyed: false, defaultEncoding: 'utf8', awaitDrain: 0, readingMore: true, decoder: null, encoding: null }, readable: true, domain: null, _events: [Object: null prototype] { end: [Function: resetHeadersTimeoutOnReqEnd] }, _eventsCount: 1, _maxListeners: undefined, socket: Socket { connecting: false, _hadError: false, _handle: [TCP], _parent: null, _host: null, _readableState: [ReadableState], readable: true, domain: null, _events: [Object], _eventsCount: 8, _maxListeners: undefined, _writableState: [WritableState], writable: true, allowHalfOpen: true, _sockname: null, _pendingData: null, _pendingEncoding: '', server: [Server], _server: [Server], timeout: 120000, parser: [HTTPParser], on: [Function: socketOnWrap], _paused: false, _httpMessage: [ServerResponse], _peername: [Object], [Symbol(asyncId)]: 65475, [Symbol(lastWriteQueueSize)]:0, [Symbol(timeout)]: Timeout { _called: false, _idleTimeout: 120000, _idlePrev: [TimersList], _idleNext: [TimersList], _idleStart: 2522901, _onTimeout: [Function: bound ], _timerArgs: undefined, _repeat: null, _destroyed: false, [Symbol(unrefed)]: true, [Symbol(asyncId)]: 65544, [Symbol(triggerId)]: 65478 }, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0 }, connection: Socket { connecting: false, _hadError: false, _handle: [TCP], _parent: null, _host: null, _readableState: [ReadableState], readable: true, domain: null, _events: [Object], _eventsCount: 8, _maxListeners: undefined, _writableState: [WritableState], writable: true, allowHalfOpen: true, _sockname: null, _pendingData: null, _pendingEncoding: '', server: [Server], _server: [Server], timeout: 120000, parser: [HTTPParser], on: [Function: socketOnWrap], _paused: false, _httpMessage: [ServerResponse], _peername: [Object], [Symbol(asyncId)]: 65475, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: Timeout { _called: false, _idleTimeout: 120000, _idlePrev: [TimersList], _idleNext: [TimersList], _idleStart: 2522901, _onTimeout: [Function: bound ], _timerArgs: undefined, _repeat: null, _destroyed: false, [Symbol(unrefed)]: true, [Symbol(asyncId)]: 65544, [Symbol(triggerId)]: 65478 }, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0 }, httpVersionMajor: 1, httpVersionMinor: 1, httpVersion: '1.1', complete: true, headers: { host: 'gitpod.app', 'x-real-ip': '10.42.2.0', 'x-forwarded-for': '192.168.100.1, 10.42.2.0', 'x-forwarded-proto': 'http', 'x-forwarded-host': 'gitpod.app:80', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15', accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-us', cookie: '_gitpod_app_=s%3A2330d830-be17-45cc-a173-24c12dcf19f9.wmBkIQTEA2hWMP9beB3t21Yl6jPIk; user-platform=5d6edc71-58e5-4dd1-a0d1-f621eed4b59b', referer: 'https://github.ninja/', 'x-forwarded-port': '443', 'x-forwarded-server': 'traefik.cypress.github.ninja' }, rawHeaders: [ 'Host', 'gitpod.app', 'X-Real-IP', '10.42.2.0', 'X-Forwarded-For', '192.168.100.1, 10.42.2.0', 'X-Forwarded-Proto', 'http', 'X-Forwarded-Host', 'gitpod.app:80', 'User-Agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15', 'Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Encoding', 'gzip, deflate, br', 'Accept-Language','en-us','Cookie', '_gitpod_app_=s%3A2330d830-be17-45cc-a173-24c12dcf19f9.wmBkIQTEA2hWMP9beB3t21Yl6jPIk; user-platform=5d6edc71-58e5-4dd1-a0d1-f621eed4b59b', 'Referer', 'https://github.ninja/', 'X-Forwarded-Port','443','X-Forwarded-Server','traefik.cypress.github.ninja' ], trailers: {}, rawTrailers: [], aborted: false, upgrade: false, url: '/auth/github/callback?code=19edddbb4a8c', method: 'GET', statusCode: null, statusMessage: null, client: Socket { connecting: false, _hadError: false, _handle: [TCP], _parent: null, _host: null,_readableState: [ReadableState], readable: true, domain: null, _events: [Object], _eventsCount: 8,_maxListeners: undefined, _writableState: [WritableState], writable: true, allowHalfOpen: true, _sockname: null, _pendingData: null, _pendingEncoding: '', server: [Server], _server: [Server], timeout: 120000, parser: [HTTPParser], on: [Function: socketOnWrap], _paused: false, _httpMessage: [ServerResponse], _peername: [Object], [Symbol(asyncId)]: 65475, [Symbol(lastWriteQueueSize)]: 0, [Symbol(timeout)]: Timeout { _called: false, _idleTimeout: 120000, _idlePrev: [TimersList], _idleNext: [TimersList], _idleStart: 2522901, _onTimeout: [Function: bound ], _timerArgs: undefined,_repeat: null, _destroyed: false, [Symbol(unrefed)]: true, [Symbol(asyncId)]: 65544, [Symbol(triggerId)]: 65478 }, [Symbol(kBytesRead)]: 0, [Symbol(kBytesWritten)]: 0 }, _consuming: false, _dumped: false, next: [Function: next],baseUrl: '', originalUrl: '/auth/github/callback?code=19edddbb4a8c', _parsedUrl: Url { protocol: null, slashes: null, auth: null, host: null, port: null, hostname: null, hash: null, search: '?code=19edddbb4a8c', query: 'code=19edddbb4a8c', pathname: '/auth/github/callback', path: '/auth/github/callback?code=19edddbb4a8c', href: '/auth/github/callback?code=19edddbb4a8c', _raw: '/auth/github/callback?code=19edddbb4a8c' }, params: {}, query: { code: '19edddbb4a8c' }, res: ServerResponse { domain: null, _events: [Object], _eventsCount: 1, _maxListeners: undefined, output: [], outputEncodings: [], outputCallbacks: [], outputSize: 0, writable: true, _last: false, chunkedEncoding: false, shouldKeepAlive: true, useChunkedEncodingByDefault: true, sendDate: true, _removedConnection: false, _removedContLen: false, _removedTE: false, _contentLength: null, _hasBody: true, _trailer: '', finished: false, _headerSent: false, socket: [Socket], connection: [Socket], _header: null, _onPendingData: [Function: bound updateOutgoingData], _sent100: false, _expect_continue: false, end: [Function: end], req: [Circular], locals: [Object: null prototype] {}, writeHead: [Function: writeHead], [Symbol(isCorked)]: false, [Symbol(outHeadersKey)]: [Object] }, body: {}, _parsedOriginalUrl: Url { protocol: null, slashes: null, auth: null, host: null, port: null, hostname: null, hash: null, search: '?code=19edddbb4a8c', query: 'code=19edddbb4a8c', pathname: '/auth/github/callback', path: '/auth/github/callback?code=19edddbb4a8c', href: '/auth/github/callback?code=19edddbb4a8c', _raw: '/auth/github/callback?code=19edddbb4a8c' }, sessionStore: MySQLStore { connection: [Pool], options: [Object], generate: [Function], _events: [Object], _eventsCount: 2, _expirationInterval: Timeout { _called: true, _idleTimeout:900000, _idlePrev: [TimersList], _idleNext: [TimersList], _idleStart: 1803039, _onTimeout: [Function], _timerArgs: undefined, _repeat: 900000, _destroyed: false, [Symbol(unrefed)]: false, [Symbol(asyncId)]: 229, [Symbol(triggerId)]: 125 } }, sessionID: '233030-be17-45cc-a173-24c1219f9', session: Session { cookie: [Object], authBag: [Object] }, _passport: { instance: [Authenticator] }, route: Route {path: '/auth/github/callback', stack: [Array], methods: [Object] } }, 'login-flow': true, clientInfo: { ua: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15', fingerprint: 'b86c2eb8f6d802f9e69216b7f1764a1a4197985c62bbb558' } }"
}

What version of Gitpod Self-Hosted do you run? The error message is referencing code that is not included in the latest version. Have you updated recently or is this an old version?

And the reference to email-domain-filter-db-impl shows, that this is not the current Gitpod Core state.

@astuffedtiger, please try to reinstall if that works for you.

Okay, that would definitely make sense. I think the last time I did this, I cloned the gitpod-io/self-hosted repo bare, then mirrored it to my GitHub Enterprise install. I did actually try to run a git pull upstream but I guess it didn’t work the way I wanted it too. I will try a full proper reinstall shortly. Thanks @svenefftinge and @AlexTugarev!

Awesome! That totally worked!

Thanks again Sven and Alex!