services.AddHangfire(x => x.UseCouchbaseStorage(configuration, "familyPhotos_hangfire"));
Posts tagged with 'ASP.NET'
Kevin Griffin is using SignalR to update web pages live. This episode is not sponsored! Want to be a sponsor? You can contact me or check out my sponsorship gig on Fiverr
Show Notes:
- It's been a long time since Kevin Griffin has been on the show. Check out 13 Kevin Griffin on Twilio
- Make sure to check out Kevin's entry on the Second Annual C# Advent
- AJAX was coined in 2005
- Some techniques we didn't cover: the Forever Frame, Server-Sent Events, also explained in a Kevin Griffin blog post on SignalR Transports
- Discourse
- Azure SignalR Service
- Swift Kick
Want to be on the next episode? You can! All you need is the willingness to talk about something technical.
Chris Woodruff is building web APIs with ASP.NET Core. This episode is sponsored by Ivacy.
Show Notes:
-
IIS web server
-
Check out episode 94 with Jeremy Miller on Lamar for more discussion about dependency injection
-
Check out episode 22 with J. Tower on .NET Core for more about .NET Standard.
-
Chris’s baseball API
-
InfoQ stuff:
-
Project Rider from JetBrains
Want to be on the next episode? You can! All you need is the willingness to talk about something technical.
Want to be a sponsor? Check out my sponsorship gig on Fiverr
Merry Christmas! This is the last day of the C# Advent. Make sure to check out all of the other great posts from 2017 and 2018. If you want to be involved next year, look for C# Advent 2019 author sign ups at the end of October 2019, and look for blog posts to start showing up on December 1st, 2019.
What is a background job?
A background job is some code that runs apart from the normal flow of your program. It could be run asynchronously and/or on another thread. As an ASP.NET MVC developer, I tend to think of it as any task that runs outside of an MVC action being invoked.
There’s two kinds of background jobs that I’m aware of:
-
Scheduled - a task that runs every N minutes, or every Y hours, etc. This is what I’m going to show in this post today. It’s great for making periodic checks, ingesting data from some other source, etc.
-
Fire and forget - Some other piece of code kicks off a process to run in the background. It doesn’t block the code (fire), and the code doesn’t wait for a response (forget). This is great for potentially time consuming operations like checking inventory, sending emails, etc, that you don’t need a user to wait for.
What you usually need to do to create background jobs
In my experience, I’ve seen background jobs take a few different forms.
-
Separate Windows service (or Linux daemon, whatever). A console/service program that’s running in addition to your ASP.NET program. This works fine for scheduled jobs.
-
Queueing mechanisms like Kafka or Rabbit. The ASP.NET program will put messages into these queues, which will then be processed by some other program. This is fine for fire-and-forget.
-
Background jobs running within the ASP.NET process itself. In my experience, I’ve used Quartz.NET, which can run within the ASP.NET process. There’s also FluentScheduler (which I’ve not used, and doesn’t seem to come with database integration out of the box?)
With all these options in the past, I’ve experienced deployment difficulties. The wrong version of the service gets deployed, or isn’t running, or fails silently, or needs to be deployed on multiple servers in order to provide scalability/availability etc. It’s totally possible to overcome these challenges, of course. (I should also note that in my experience with Quartz.NET, I never used it in embedded form, and the last time I used it was probably 6+ years ago).
But if I just need a handful of background jobs, I’d much rather just make them part of the ASP.NET system. Yes, maybe this goes against the whole 'microservice' idea, but I don’t think it would be too hard to refactor if you decided you need to go that route. I solve my deployment problems, and as you’ll see with Hangfire (with Couchbase), it’s very easy to scale.
How hangfire works
You can find more details and documentation about Hangfire at Hangfire.io. Really, there are only three steps to setting up Hangfire with ASP.NET Core:
-
Tell ASP.NET Core about Hangfire
-
Tell Hangfire which database to use
-
Start firing off background jobs
In Startup.cs, in the ConfigureServices
method:
Then, in Startup.cs, in the Configure
method:
app.UseHangfireServer();
I’m using Couchbase in this example, but there are options for SQL Server and other databases too. I happen to think Couchbase is a great fit, because it can easily horizontally scale to grow with your ASP.NET Core deployments. It also has a memory-first architecture for low latency storage/retrieval of job data. Generally speaking, even if you use SQL Server as your "main" database, Couchbase makes a great companion to ASP.NET or ASP.NET Core as a cache, session store, or, in this case, backing for Hangfire.
The configuration
variable is to tell Hangfire where to find Couchbase:
var configuration = new ClientConfiguration
{
Servers = new List<Uri> { new Uri("http://localhost:8091") }
};
configuration.SetAuthenticator(new PasswordAuthenticator("hangfire", "password"));
(In my case, it’s just running locally).
Steps 1 and 2 are down. Next, step 3 is to create some background jobs for Hangfire to process. I’ve created an ASP.NET Core app to assist me in the cataloging of all my family photographs. I want my application to scan for new files every hour or so. Here’s how I create that job in Hangfire:
RecurringJob.AddOrUpdate("photoProcessor", () => processor.ProcessAll(), Cron.Hourly);
Note that I didn’t have to implement an IJob
interface or anything like that. Hangfire will take any expression that you give it (at least, every expression that I’ve thrown at it so far).
Step 3 done.
Hangfire is just a NuGet package and not a separate process. So no additional deployment is needed.
How do I know it’s working?
Another great thing about Hangfire is that is comes with a built-in dashboard for the web. Back in Startup.cs, in Configure
, add this code:
app.UseHangfireDashboard("/hangfire", new DashboardOptions
{
Authorization = new[] {new HangfireAuthorization()}
});
I’m using my own HangfireAuthorization
implementation because Hangfire only gives permission to local users by default.
Then, you get a nice dashboard right out of the box, showing you a realtime and history graph.
From this dashboard, you can also look at a more detailed history of what’s run and what’s failed.
You can even kick off recurring jobs manually.
This is only the start
If you’re thinking about adding background jobs to your ASP.NET Core solution, why not give Hangfire a try?
Some more things for you to explore:
-
Scaling: every ASP.NET Core site that gets deployed with Hangfire that points to the same database will be able to process jobs too. As your ASP.NET Core site scales out, hangfire scales out with it. This is another reason that Couchbase is a good fit, because it’s also easy to scale out as your site grows.
-
Cloud: If you are deploying your site as an app service, note that Azure will shut down ASP.NET processes if they haven’t been used in a while. This means Hangfire will shut down with them. There are a couple of ways to deal with this. Check out the Hangfire documentation.
-
Retries: Hangfire will retry failed jobs. Design your background job code to expect this.
-
Hangfire Pro: The commercial version of Hangfire is called Hangfire.Pro, and it comes with some interesting looking batch capabilities. I’ve not needed any of this functionality yet, but for more advanced cases you might need this.
-
Couchbase: a NoSQL data platform that has a built-in memory-first cache layer, SQL support, text search, analytics, and more. There are lots of options for working with Couchbase in .NET. For this post, I used the Hangfire.Couchbase library (available on NuGet).
Ed Charbeneau is creating and using ASP.NET tag helpers. This episode is sponsored by Smartsheet.
Show Notes:
-
Doom and web page size: I think this was originally pointed out by Ronan Cremin
-
(Doom is a 1993 PC game, here’s a video of Doom in action)
-
I also tweeted sarcastically about page footprint and client-side rendering recently.
-
-
Progress Telerik tools
-
Vue Vixens (I couldn’t find their Rick & Morty example though)
-
Docs: Tag Helpers
-
Scott Addie is on Twitter
-
-
Demos: Telerik ASP.NET Core demos
-
Eat Sleep Code podcast (also on Soundcloud)
Want to be on the next episode? You can! All you need is the willingness to talk about something technical.
Music is by Joe Ferg, check out more music on JoeFerg.com!
This is a repost that originally appeared on the Couchbase Blog: Geospatial Search with ASP.NET Core, Aurelia, and Google Maps.
Geospatial search is now fully supported in Couchbase Server 5.5. Check out the Couchbase Server 5.5 announcement, and download the developer build for free right now.
In this post, I’m going to demonstrate the geospatial search capabilities of Couchbase Full Text Search by creating a web-based UI that performs searches. Whenever I think of geospatial searches, I think about Yelp, which is great at helping me find restaurants in a specific area.
So I’m going to have a little fun and create a very bare-bones version of Yelp, but only for hotels.
If you want to follow along, the full source code is available on Github.
Getting set up
Here are the steps I took to create a new project before I started writing code.
-
At the command line:
dotnet new aurelia
. This assumes that you have .NET Core installed. Note that Geospatial Search is not a .NET-only feature: you can use it with the other Couchbase SDKs like Node.js, Java, etc. It also assumes that you’ve installed a SPA template for Aurelia. You can also go with Angular or React if you’d like, but I really like Aurelia, and I think you should give it a chance. -
The above command will create a shell of an ASP.NET Core project. In this blog post, I’m not going to use Razor. I’m just using ASP.NET as a backend for REST API endpoints.
-
npm install aurelia-google-maps
. You don’t have to use this, but the aurelia-google-maps plugin will make it easy for me to interact with Google Maps in my app. -
I opened this project in Visual Studio 2017. I added Couchbase.Extensions.DependencyInjection with NuGet. You don’t have to use this extension but it makes things easier.
-
I installed Couchbase Server 5.5, including the Full Text Search service. I setup the travel-sample bucket. I created a user "matt" with full access to that bucket.
Create a Geospatial Index
Before building the ASP.NET backend, we need to create a geospatial index in Couchbase Server. Once you log in, click "Search" on the menu (it’s under "Workbench"). Click "Add Index" to get started.
I named my index "mygeoindex". I selected travel-sample as the bucket to index.
In "Type Mappings", I uncheck the default. I add a new type mapping with a type name of "hotel". Every hotel document in "travel-sample" has a type with a value of "hotel". Check the "only index specified fields" box.
I’m going to add two child fields. One is "geo", which contains the geospatial coordinates inside a hotel document. Make sure to select "geopoint" as the type. The other is "name", which will be the name of the hotel. I choose to "store" each of these: it will make the index larger, but I can avoid a secondary lookup if I store the information in the index.
Important Note: There is a bug (NCBC-1651) in the current release of the .NET SDK that will cause an error if you try to read from a geopoint field. In the code samples, I’ve created a workaround: I don’t actually get the geo & name fields from the search index. I instead use the document key returned by search to make a secondary "get" call and get the full document. Keep in mind this is still a technique you may want to consider if you want to keep the size of your index down. This bug has already been fixed and will be in a future release. Such is the peril of being on the cutting edge!
That’s all there is to it. Click "Create Index". Watch the "indexing progress" on the next screen until it gets to 100% (it should not take very long, assuming you remembered to uncheck "default").
ASP.NET Core REST Endpoints
Next, let’s move over to ASP.NET. I’ll create two endpoints. One endpoint will demonstrate the bounding box search method, and the other will demonstrate the distance search method.
I’ll need a Couchbase bucket object to execute the queries. Follow the examples in my blog post about dependency injection or check out the source code on Github if you’ve never done this before.
Bounding Box
A "bounding box" search means that you define a box on a map, and you want to search for points of interest that are inside of that box. You only need two points to define a box: the top right corner coordinates and the bottom left corner coordinates. (Coordinates are latitude and longitude).
public class BoxSearch
{
public double LatitudeTopLeft { get; set; }
public double LongitudeTopLeft { get; set; }
public double LatitudeBottomRight { get; set; }
public double LongitudeBottomRight { get; set; }
}
To create a bounding box geospatial query, use the GeoBoundingBoxQuery
class available in the .NET SDK. I’ll do this inside of a POST method with the above BoxSearch
class as a parameter.
[Route("api/Box")]
[HttpPost]
public IActionResult Box([FromBody] BoxSearch box)
{
var query = new GeoBoundingBoxQuery();
query.TopLeft(box.LongitudeTopLeft, box.LatitudeTopLeft);
query.BottomRight(box.LongitudeBottomRight, box.LatitudeBottomRight);
var searchParams = new SearchParams()
// .Fields("geo", "name") // omitting because of bug NCBC-1651
.Limit(10)
.Timeout(TimeSpan.FromMilliseconds(10000));
var searchQuery = new SearchQuery
{
Query = query,
Index = "mygeoindex",
SearchParams = searchParams
};
var results = _bucket.Query(searchQuery);
// ... snip ...
All I need to return from this endpoint is a list of the results: each hotel’s coordinates and the hotel’s name & location. I created a GeoSearchResult
class for this.
public class GeoSearchResult
{
public double Latitude { get; set; }
public double Longitude { get; set; }
public InfoWindow InfoWindow { get; set; }
}
public class InfoWindow
{
public string Content { get; set; }
}
I’ve constructed this class to match the Google Maps plugin that I’ll be using later.
Finally, I’ll use this class to return some results from the endpoint.
// ... snip ...
var list = new List<GeoSearchResult>();
foreach (var hit in results.Hits)
{
// *** this part shouldn't be necessary
// the geo and name should come with the search results
// but there's an SDK bug NCBC-1651
var doc = _bucket.Get<dynamic>(hit.Id).Value;
// ****************
list.Add(new GeoSearchResult
{
Latitude = doc.geo.lat,
Longitude = doc.geo.lon,
InfoWindow = new InfoWindow
{
Content = doc.name + "<br />" +
doc.city + ", " +
doc.state + " " +
doc.country
}
});
}
return Ok(list);
}
Distance Search
A "distance" search is another way to perform geospatial queries. This time, instead of a box, it will be more like a circle. You supply a single coordinate, and a distance. The distance will be the radius from that point.
public class PointSearch
{
public double Latitude { get; set; }
public double Longitude { get; set; }
public int Distance { get; set; }
// miles is being assumed as the unit
public string DistanceWithUnits => Distance + "mi";
}
I’m defaulting it to miles, but certainly you can use kilometers instead, or present the option in the UI.
The endpoint will be very similar to the bounding box endpoint, except that it uses GeoDistanceQuery
.
[Route("api/Point")]
[HttpPost]
public IActionResult Point([FromBody] PointSearch point)
{
var query = new GeoDistanceQuery();
query.Latitude(point.Latitude);
query.Longitude(point.Longitude);
query.Distance(point.DistanceWithUnits);
var searchParams = new SearchParams()
// .Fields("geo", "name") // omitting because of bug NCBC-1651
.Limit(10)
.Timeout(TimeSpan.FromMilliseconds(10000));
var searchQuery = new SearchQuery
{
Query = query,
Index = "mygeoindex",
SearchParams = searchParams
};
var results = _bucket.Query(searchQuery);
var list = new List<GeoSearchResult>();
foreach (var hit in results.Hits)
{
// *** this part shouldn't be necessary
// the geo and name should come with the search results
// but there's an SDK bug NCBC-1651
var doc = _bucket.Get<dynamic>(hit.Id).Value;
// ****************
list.Add(new GeoSearchResult
{
Latitude = doc.geo.lat,
Longitude = doc.geo.lon,
InfoWindow = new InfoWindow
{
Content = doc.name + "<br />" +
doc.city + ", " +
doc.state + " " +
doc.country
}
});
}
return Ok(list);
}
At this point, you can start testing these endpoint with Postman or Fiddler if you’d like. But it will be so much nice to see this on a map.
Auerlia and Google Maps
In Aurelia, I’ve created two components: geosearchbox and geosearchpoint.
Each of them will have a Google Maps component that the user can interact with. These maps will be centered on San Francisco, because that’s where a lot of the hotels in "travel-sample" are located.
Bounding Box search component
The google-map`
component has a map-click.delegate that will will fire whenever the users clicks on the map. In geosearchbox.html:
<google-map
if.bind="markers"
map-click.delegate="clickMap($event)"
latitude="37.780986253433895"
longitude="-122.45291600632277"
zoom="12"
markers.bind="markers">
</google-map>
markers
is simply an array containing coordinates of search results that should appear on the map. Initially it will be empty.
When the user first clicks the map, this will set the first coordinate (top left) in the form. In geosearchbox.ts:
public clickMap(event : any) {
var latLng = event.detail.latLng,
lat = latLng.lat(),
lng = latLng.lng();
// only update top left if it hasn't been set yet
// or if bottom right is already set
if (!this.longitudeTopLeft || this.longitudeBottomRight) {
this.longitudeTopLeft = lng;
this.latitudeTopLeft = lat;
this.longitudeBottomRight = null;
this.latitudeBottomRight = null;
} else {
this.longitudeBottomRight = lng;
this.latitudeBottomRight = lat;
}
}
Then, click another spot on the map. This will set the second coordinate (bottom right).
My implementation is very bare bones. No fancy graphics and no validation of the second coordinate being to the bottom right of the first. The fields on a form will simply be populated with the latitude and longitude. In geosearchbox.html:
<p>
Bounding box search:
<br />
Latitude (top left):
<input type="text" value="${ latitudeTopLeft }" />
Longitude (top left):
<input type="text" value="${ longitudeTopLeft }" />
<br />
Latitude (bottom right):
<input type="text" value="${ latitudeBottomRight }" />
Longitude (bottom right):
<input type="text" value="${ longitudeBottomRight }" />
<br />
<input
if.bind="latitudeTopLeft && latitudeBottomRight"
click.trigger="searchClick()"
type="button"
name="search"
value="Search" />
</p>
Once you’ve selected two coordinates, a search button will appear. Click that to post these coordinates to the endpoint created earlier, and it will trigger the searchClick()
method as seen in geosearchbox.ts:
public searchClick() {
let boxSearch = {
latitudeTopLeft: this.latitudeTopLeft,
longitudeTopLeft: this.longitudeTopLeft,
latitudeBottomRight: this.latitudeBottomRight,
longitudeBottomRight: this.longitudeBottomRight
};
console.log("POSTing to api/Box: " + JSON.stringify(boxSearch));
this.http.fetch('api/Box', { method: "POST", body: json(boxSearch) })
.then(result => result.json() as Promise<any[]>)
.then(data => {
this.markers = data;
});
}
When Aurelia, Google Maps, ASP.NET Core, and Couchbase all work together, it looks like this:
Distance Search
Implementing the "distance" geostatial query will be similar to the bounding box UI. This time, you only need to click a single point on the map. But, you will need to type in a distance (in miles).
The google-map
component will look identical. The clickMap
function is different:
public clickMap(event: any) {
var latLng = event.detail.latLng,
lat = latLng.lat(),
lng = latLng.lng();
this.longitude = lng;
this.latitude = lat;
}
Specify a distance (in miles), and then click 'search' to make a POST request to the endpoint we wrote earlier.
geosearchbox.html:
<p>
Distance search:
<br />
Latitude: <input type="text" value="${ latitude }" />
Longitude: <input type="text" value="${ longitude }" />
<br />
Distance (miles): <input type="text" value="${ distance }" />
<br />
<input if.bind="latitude" click.trigger="searchClick()" type="button" name="search" value="Search" />
</p>
geosearchbox.ts:
public searchClick() {
let pointSearch = {
latitude: this.latitude,
longitude: this.longitude,
distance: this.distance
};
console.log("POSTing to api/Point: " + JSON.stringify(pointSearch));
this.http.fetch('api/Point', { method: "POST", body: json(pointSearch) })
.then(result => result.json() as Promise<any[]>)
.then(data => {
this.markers = data;
});
}
}
Below is a clip of the search in motion. Note how the results change as I move the coordinate around.
Summary
With Couchbase’s built-in geospatial indexing and search feature, all the math and the searching is delegated to the Couchbase Data Platform. So you can focus on building a killer UI (better than mine anyway) and rock-solid business logic.
Be sure to check out the documentation for a complete overview of the geospatial capabilities of Couchbase.
If you need help or have questions, please check out the Couchbase Server forums, and if you have any questions about the Couchbase .NET SDK, check out the .NET SDK forums.
If you’d like to get in touch with me, please leave a comment or find me on Twitter @mgroves.