When telescope is enabled, one of the routes in our app gives an nginx 502 bad gateway error. We experience this on a forge server
and locally
on valet. Without telescope, this route works fine. It is fairly resource intensive, but it does work. Tried (unsuccessfully) to debug this in various ways but haven't come up with much. Strangely, commenting out everything in the
register()
method of my app's
TelescopeServiceProvider
doesn't change anything. Toggling the
TELESCOPE_ENABLED
env variable does fix the issue though.
The data I have is a collection of 6 eloquent models, each with 7 relations loaded
some of which also load additional relations themselves
. I suspect something in this nesting or total string/data length is causing the issue with the json encoding. Will report back as I discover more.
'properties' => json_decode(json_encode($value->take(3)), true),
This is not a solution, I just did it as an example. If I increase the number to 5+, it causes nginx to return a 502 again. Perhaps the array/object is just too big to serialize in this case?
Happy to show you on a google Hangout if you'd like to pair up to investigate. Is trimming / limiting the data that gets sent to json_encode in certain cases something that you'd consider adding? Can give it a go and submit a PR if so.
Oh... and Telescope is awesome by the way! Thanks for sharing such a great package with the community!
I believe this comes down to the view having too many nested relations that are being serialized. Even calling toArray() on the $value is causing the 502. Thoughts on how we might limit the length / size of view data telescope records?
Noticed there is a function (for string-based responses) called contentWithinLimits.. Something like that, but for collections of models or view data in general?
Yeah... from your perspective I'm sure it's difficult to help much given the info I've provided so far. This is a tricky one to figure out, so giving more info is difficult! Regardless, I appreciate your patience and constructive comments, thank you.
Even trying to do something like dd(json_encode($value)) will cause it to 502 (and the PHP fpm process to restart)... so printing it out unfortunately won't work. I get the same 502 result if I try something like dd($value->toArray()). Simply doing dd($value) works though, so it tells me it's in the serialization somewhere? I had some parts of it dumping out and printing the items from $value (remember, this is a collection of models in my case right now) one by one. It silently failed in similar spots, but not always in the same place... as if PHP couldn't handle that big thing in memory or something.... but there are no PHP errors about memory or execution time :(
Anyway, best I can tell, it is the size of the content housed in the serialized models. It's simply too much for json_encode to handle. These models in particular have several layers of relations which also get serialized too. Wondering if we can somehow limit the nesting of relations before it gets passed to json_encode?
It may be difficult to provide a repo with an example of this, but I'll give it a shot!
@austenc what if you put it inside a try...catch? something like:
protected function extractDataFromView($view)
return collect($view->getData())->map(function ($value) {
if ($value instanceof Model) {
return FormatModel::given($value);
} elseif (is_object($value)) {
return [
'class' => get_class($value),
'properties' => json_decode(json_encode($value), true),
}catch(\Throwable $e){
return [
'class' => get_class($value),
} else {
return json_decode(json_encode($value), true);
})->toArray();
Aha! Finally found something. It seems the PHP / Nginx do some really strange things when hitting memory limits. I believe this would normally be an out of memory exception or an SQL error, but in some cases the php-fpm process itself restarts before it gets to that point.
Now, while not a 502 error exactly, I was able to get telescope to throw some MySQL errors if too big a dataset is passed in to a view.
String data, right truncated: 1406 Data too long for column 'content'
I realize this has been brought up before, but I believe that was in a different context. This seems to still point to the lines in RequestWatcher I was referencing above.
Please see reproduction repo here:
https://github.com/austenc/telescope-bug
With that, you should be able to load the homepage. It will load fine in a browser, but if you check the project's laravel.logfile you'll see the SQL error. Seems we need to check content length of the view data also?
Please let me know if I can help or provide any other information!
Also running Docker with Nginx, and as soon as I enable Telescope, I get a 500 Internal Server error. This is the output I'm able to capture:
[error] 8#8: *52 FastCGI sent in stderr: "PHP message: PHP Fatal error: Uncaught Error: Call to undefined function Moontoast\Math\bcadd() in /var/www/vendor/moontoast/math/src/Moontoast/Math/BigNumber.php:506
Stack trace:
#0 /var/www/vendor/moontoast/math/src/Moontoast/Math/BigNumber.php(62): Moontoast\Math\BigNumber->setValue('154774728994203')
#1 /var/www/vendor/ramsey/uuid/src/Converter/Number/BigNumberConverter.php(49): Moontoast\Math\BigNumber->__construct('154774728994203')
#2 /var/www/vendor/ramsey/uuid/src/Generator/CombGenerator.php(72): Ramsey\Uuid\Converter\Number\BigNumberConverter->toHex('154774728994203')
#3 /var/www/vendor/ramsey/uuid/src/UuidFactory.php(235): Ramsey\Uuid\Generator\CombGenerator->generate(16)
#4 /var/www/vendor/laravel/framework/src/Illuminate/Support/Str.php(552): Ramsey\Uuid\UuidFactory->uuid4()
#5 /var/www/vendor/laravel/telescope/src/IncomingEntry.php(74): Illuminate\Support\Str::orderedUuid()
#6 /var/www/vendor/laravel/telescope/src/IncomingExceptionEntry.php(27): Laravel\Telescope\IncomingEntry->__construct(...
PHP message: PHP Fatal error: Uncaught Error: Call to undefined function Moontoast\Math\bcadd() in /var/www/vendor/moontoast/math/src/Moontoast/Math/BigNumber.php:506
Stack trace:
#0 /var/www/vendor/moontoast/math/src/Moontoast/Math/BigNumber.php(62): Moontoast\Math\BigNumber->setValue('154774729014768')
#1 /var/www/vendor/ramsey/uuid/src/Converter/Number/BigNumberConverter.php(49): Moontoast\Math\BigNumber->__construct('154774729014768')
#2 /var/www/vendor/ramsey/uuid/src/Generator/CombGenerator.php(72): Ramsey\Uuid\Converter\Number\BigNumberConverter->toHex('154774729014768')
#3 /var/www/vendor/ramsey/uuid/src/UuidFactory.php(235): Ramsey\Uuid\Generator\CombGenerator->generate(16)
#4 /var/www/vendor/laravel/framework/src/Illuminate/Support/Str.php(552): Ramsey\Uuid\UuidFactory->uuid4()
#5 /var/www/vendor/laravel/telescope/src/IncomingEntry.php(74): Illuminate\Support\Str::orderedUuid()
#6 /var/www/vendor/laravel/telescope/src/IncomingExceptionE
So it looks like you need to rebuild PHP with BC Math support:
http://php.net/manual/en/bc.installation.php
Problem is, the version of PHP I have on Windows and on Docker (with Ubuntu) both support this function, and I can use Tinker or Git Bash to run the command.
Some more updates. If I replace the bcadd() function in that file with the alternative one on the documentation page, it works but then has problems with bcdiv(). I guess I could just keep replacing those functions, but that seems pointless.
Last update, if you edit the Telescope config file, and disable the RequestWatcher watcher, I can get the site to come up now.
@divspace install the mbstring bcmath extensions for your docker php and problem solved.
my dockerfile:
FROM php:7.2.8-fpm-alpine3.7
RUN apk add --no-cache --update --virtual buildDeps autoconf automake make gcc g++\
&& pecl install redis \
&& pecl install xdebug \
&& docker-php-ext-enable redis xdebug \
&& docker-php-ext-install pdo pdo_mysql json mbstring bcmath \
&& docker-php-ext-enable pdo pdo_mysql json mbstring bcmath \
&& apk del buildDeps autoconf automake make gcc g++
So there are no plans to take action on this then? Bummer as it seems quite
a few people have bumped into it with bigger data sets.
