You guessed it — frequently asked questions!
Skylight plays nice with other profilers and monitoring solutions, including New Relic. Many of our customers run multiple tools with success! Go ahead and compare :)
Skylight measures the time spent in Rails actions, and most of the time, your requests end up making their way into Rails actions eventually. Once a request makes its way to an endpoint, Skylight tags the request with the endpoint’s name. However, it is possible for Rack middleware to intercept the request and return a response without delegating to an endpoint. When this happens, Skylight uses the generic “Rack” name for the request, and in practice, these requests tend to be very fast. Upgrading to the 1.4 agent will split these out by the name of the
middleware that handled the request.
There are a number of different reasons why you might see “Rack” listed in your Endpoints List. For example,
Rack::Cache may be intercepting your request when your app gets a cache hit. Another example is
ActionDispatch::Static, which is an interesting case. This middleware handles requests in the /public folder and acts as an endpoint for static files. While it is not a Rails action, it does behave quite similar to a Rails action: it uniquely handles a particular URL and returns a result.
In Rails apps, “rack.request” will usually be the first item listed in the Event Sequence. This event represents the time spent before the actual Rails controller action is hit. Usually this event has minimal self-time, but some middleware may cause slowdowns in the “rack.request” event. Upgrading to the 1.4 agent will show detail about Rails middleware.
Skylight follows generally acceptable security practices. Most importantly, we don’t store any sensitive data ourselves.
When you install the Skylight agent in your app, it will send the following information to our servers:
Nope. All performance metrics are scrubbed of private data by the agent running on your server before anything is ever sent our way. Skylight doesn’t collect sensitive data like request param or database query values.
In fact, scrubbing parameters and values from the data your app sends is what allows Skylight to aggregate your data, providing you with an accurate view of your app’s performance on the whole.
For example, Skylight will remove variables from your SQL queries and display aggregated queries in the UI along with average durations and allocations:
If you signed up for Skylight using GitHub, you may have seen this before:
We don’t really need write access to your repos, nor do we use it. We just want to check all the repos you have access to (public and private) to see if there are any apps on Skylight that are attached to those repos so we can give you automatic access.
Unfortunately, GitHub OAuth does not offer read-only access to public and private user repos (you can see the scopes they offer here, so we have to ask for read/write access even though we’re really just reading.
We also request read-only access to your organizations. This is so you can choose a repo belonging to a specific organization in order to attach it to your Skylight app. There is more information about this here.
If someone is added to a repo, they should have access to the app on Skylight as soon as they log in with GitHub.
We run a daily check to make sure everyone’s access is exactly as it should be. If someone’s repo access is removed, their Skylight access will be revoked when we run that check.
The easiest way to gain access to Skylight’s UI beta features is to become a Skylight Insider. It’s super easy. Just head to the Labs page and click “SUBSCRIBE” to cement your status. Once you’ve become an Insider, you can toggle beta feature flags on and off to your heart’s content.
Read more about our feature toggles here.
Message firstname.lastname@example.org or contact us via the in-app messenger to see if your app is a good candidate for helping us test new agents and new agent features.
While we don’t yet give you up to the minute alerts, you can get insight into your app’s weekly changes by signing up for Trends emails in your account settings. Down the road, we may add more immediate alerting when significant slowdowns occur in your app.
Soon! We’ve got work underway to make this happen. Sign up for our Insider emails in your account settings to get the latest scoop on development.
Background job instrumentation is a high priority for us. The technical aspect of it isn’t too hard, but the big blocker is the UI. We don’t just want to throw the background jobs into the regular UI since in most cases, they’ll take much longer than normal requests and won’t have the same immediate impact on the end-user.
But, we ourselves use Sidekiq and agree that it would be great to have more insight into our background jobs performance.
Not yet, but it’s something we’ve got on our roadmap. Our current thought is something based around GitHub organizations. If you’ve got any thoughts on this, we’d love to hear them.
Right now we’ve chosen to focus on providing you the most useful performance information we can. That said, in the future we’d love to provide some basic notifications, especially around increased error rates. For more robust error tracking, we recommend a dedicated tool like Bugsnag.
Yes, you can, via the Trends email, which you’ll get after your first two weeks of using Skylight. The Trends email shows the last six weeks of your application’s performance.
Showing weeks and months of data on demand is not as simple as it might seem. We use a technique that many time-series databases use called pre-aggregation. This means that we store all your data collated into timeslices. Timeslices make it possible to fetch arbitrarily large time ranges or time ranges that are not simply aligned to the day or hour.
At this stage, we haven’t been pre-aggregating data past one hour sizes. This means that if we were to query for a two month period, we’d need to make 24 x 60 queries to fetch the data. This can quickly escalate to 100s of 1000s of queries for a single application.
Alas, there’s no way to set a limit or cap right now. This was possible in the very first version of Skylight, and contrary to our expectations, more people disliked it than found it helpful.
It turned out most of our users were already accustomed to usage-based services since they were already using them for things like hosting (Heroku, AWS, etc) and other parts of the stack. When we had the ability to set a limit, people would set it aggressively low in an effort to be frugal and then get annoyed by the notifications when they consistently hit those unrealistic thresholds. While we’re sure some people would still appreciate it, overall it made significantly more people unhappy than happy, so we didn’t build it into our next version.
It’s also worth mentioning that Skylight is most accurate when reviewing aggregated requests over a longer period of time, so by capping your requests and not giving Skylight all the information available, you may end up on some wild goose chases.
New Relic has lots of features we don’t have but, in our opinion, they’re not all that useful. We’ve decided to focus on making sure that everything we show you provides immediate value. Our customers find that it’s easier to make sense of our UI and to get actionable information from it. Sure, we could throw more stats in, but sometimes less is more.
Yes! You can apply for a free Skylight for Open Source account.
The goal of this program is to help encourage your contributors to find and address performance issues in your open source app, by giving them actionable insights and feedback with Skylight. To ensure we can be effective in achieving these goals, your app must meet the following requirements to qualify for the program:
Here are some specific examples:
The Odin Project and the Homebrew Formulae Browser are both examples of single-deployment apps that meet the requirements. Their source code is available on Github under an open source license. They are deployed as publicly accessible website available for anyone to use or sign up for free, other than any administration functionality.
Discourse is an example of a multi-deployment app. While its source code is available on Github under an open source license, it is up to the individual site operators to deploy their own installation of the app (either with the official managed hosting option or self-hosting).
In this case, the purpose of the program is to help the site operators find performance issues in the underlying open source software (Discourse in this example), so that they can easily report problems to the maintainers or upstream any patches.
The accessibility and purpose of the specific deployment will determine if it qualifies for the program. For example, discuss.emberjs.com will qualify as it is publicly accessible and anyone can participate in the discussions on the forum.
On the other hand, a private Discourse forum for your patreons will not qualify for the program at this time. Similarly, an Errbit instance or an open-source Slack bot deployed for your company’s internal use will also not qualify.
Finally, an open-source app that requires payment to access its features will generally not be accepted into the program at this time. Examples of this include blogs with paywalls, bitcoin exchanges and ICO websites. Companion websites for paid products, such as support websites or apps that are meant to used with a paid smartphone app, will also not qualify as they lose their primary utility without the paid product.
Skylight for Open Source is a new program that we’re still experimenting with. We reserve the right to update or clarify these guidelines, make exceptions to them, or revoke access as the program evolves.
If you have any questions, please feel free to reach out at email@example.com.
Skylight works differently than other similar products, in that we rely heavily on aggregation, both for presenting useful data in the UI and also to keep our backend scalable. We don’t keep parts of individual requests; the requests are essentially only used as “data points” to build statistical models about your app and its endpoints. For example, any SQL queries are parsed and sanitized on your server before they are sent to us. There is no way to get from the aggregated data back to an individual request.
Read more about security and Skylight.
Great question! We created the Skylight for Open Source program to make it easy for open source maintainers to share performance data with their contributors. You can read more about the Skylight for Open Source program and why it exists here.
We would love to hear your questions. Just shoot us an email at firstname.lastname@example.org or use the in-app messenger (“?” in the bottom right of the web UI). See our section on Submitting Feedback for more details.