PREFACE

CONTEXT

Wisconsin Department of Public Instruction’s (DPI) WISEdata project is about improving Federal reporting. Federal reporting can be improved by replacing individual legacy collection systems (ISES, CWCS, etc) with a cohesive interoperability collection framework. DPI has chosen the Ed-Fi interoperability framework to facilitate collecting local education agency (LEA) information. This framework will need to be implemented (if it is not already) by the LEA’s Student Information System (SIS). The SIS software will push data to DPI via a web service defined by the Ed-Fi framework (see Figure-1).

Open

Figure 1: The flow of data from an LEA to DPI via Ed-Fi web service.

PURPOSE

The purpose of this document is to describe the process of sending and receiving data using the Ed-Fi REST client against the Ed-Fi API hosted by DPI (WISEdata). The API follows REST guidelines which makes sending and receiving data relatively standard (via  HTTP verbs like GET and POST). This document also describes the authorization using OAuth 2 which is critical for connecting to the Ed-Fi API. 

Note: This information is also documented by Ed-Fi and DLP. It is strongly recommended that you review their documentation which is listed in the references sections.

USE OF THIS GUIDE

1. This document is appropriate for programmers who wish to write clients to send requests to the WISEdata via secure RESTful Ed-Fi API.
   
   

2. The use of this document assumes no particular programming language or framework. However, the programmer should be familiar with RESTful services and Basic HTTP Authentication. A brief overview of these concepts is given below. References for additional reading are also provided

DEFINITIONS, ACRONYMS, AND ABBREVIATIONS

REFERENCES

SYSTEM OVERVIEW

The Ed-Fi API exposes endpoints for submitting and retrieving educational data. Those calls are secure and synchronous (the response is immediately returned).  

PREREQUISITES 

• CREDENTIALS

1. The Ed-Fi API is service secured via OAuth 2.

2. An OAuth key and secret (per district) is needed to make REST calls.

   1. Secret 

      1. Get from DPI - it will be in the Getting Started Guide

   2. Key

      1. Get from DPI - it will be in the Getting Started Guide

3. Each OAuth credential needs to be associated with the correct district which will be done by DPI.

   1. Test district ID

      1. Get from DPI - it will be in the Getting Started Guide

   2. Test school

      1. Get from DPI - it will be in the Getting Started Guide

• TESTING TOOLS

1. A REST client which can authenticate via OAuth 2. There are many free and paid REST clients that you can use for testing. The two clients that we use are listed below. Both are free.

   1. SoapUI (version 4.6.4): Besides supporting SOAP requests, it also has a REST client. The World's Most Popular API Testing Tool | SoapUI

   2. RestClient 2: This is a FireFox extension. According to the developer,  they also have an extension for Safari and for Chrome. Developer website: RESTClient, a debugger for RESTful web services.

2. A web traffic tracer / debugger. There are a number of web debuggers out there. The REST clients above will (SoapUI or RestClient2) will show web traffic when you make a request. The web debuggers can be handy when creating request via the DPI provided WISEdata Ed-Fi REST SDK or a proprietary Ed-Fi REST client. 

   1. Fiddler is free and one of the more popular web debuggers Web Debugging Proxy and Troubleshooting Tools | Fiddler

   2. Wireshark Wireshark · Go Deep

3. WISEdata Ed-Fi REST SDK.zip which is. 

   1. The SDK was Written in C# / .NET and developed using Visual Studio 2013.

   2. The SDK has code for OAuth 2, Ed-Fi REST clients, and integration/unit tests.

   3. There is more information about the SDK in the Appendix - WISEdata Ed-Fi REST SDK 

   4. The Ed-Fi REST clients can be created manually for C# or Java using the DLP generator (see Appendix - Manually Creating the Ed-Fi REST client).

   5. It is not mandatory to use the DLP provided client. However, when developing an Ed-Fi REST client adhere to the Design Guidelines.

AUTHENTICATION - OAUTH 2

PREREQUISITES 

1. A REST client is required to make REST calls to the Ed-Fi API.

   1. The client that will publish data to the Ed-Fi API should be the SIS vendor software. The SIS implementation will likely be based on the Ed-Fi REST client (an example of the client code is listed below in Appendix).

   2. The client which is being used in the testing phase is called Swagger UI

      1. https://uaapps.dpi.wi.gov/EdFiSwagger/ 

2. A client key and secret is required to connect

   1. DPI will provide the key and secret for each LEA/District to SIS vendors.

   2. These are credentials to authenticate/authorize a client.

   3. See Figure 2 for an example. Note the key/secret pair are entered first. Next, the user has to press the “Get Token” button to get a token which is required to make REST calls.

Open

Figure 2: Swagger UI is REST client for testing Ed-Fi sandboxes.

Authorization via OAuth 2

DEFINITIONS

1. OAuth 2

   1. This is a protocol that provides clients access to server resources without including client credentials. For more information see: OAuth  

   2. For more information about Ed-Fi OAuth 2 authentication see http://www.ed-fi.org/assets/2013/11/Public-Ed-Fi-REST-API-Design-Guidelines-1.2.pdf

2. API URL

   1. The URL to the Ed-Fi API RESTful web service

   2. ex. https://edfi-rest.dpi.wi.gov/ (not a working URL)

3. Admin URL

   1. The URL to the Ed-Fi OAuth web service

   2. https://uaapps.dpi.wi.gov/EdFiWebApi 

4. Key

   1. Part of the clients credentials

   2. DPI will provide each LEA a Key

   3. ex. ekVNpbjj6SbQ

5. Secret

   1. Part of the clients credentials

   2. DPI will provide each LEA a secret

   3. ex. 9mnMlZKaswgW9RizPdEKdk

6. Token

   1. A value obtained in the OAuth protocol which is used to access the API Server

Ed-Fi Two-Legged OAuth 2 Protocol Sequence:

1. POST https://uaapps.dpi.wi.gov/EdFiWebApi/oauth/authorize  

   1. Include the paremeter “Client_id” with the clients Key value

   2. include the paremeter “Request_type” as “code”

   3. Store the response as client access code

2. POST https://uaapps.dpi.wi.gov/EdFiWebApi/oauth/token 

   1. include the parameter “Client_id” with the clients Key value

   2. include the parameter “Client_secret” with the clients secret value

   3. include the parameter “Code” with the clients access code value

   4. include the parameter “Grant_type” with the value “authorization_code”

   5. Store the response as the client access token

3. Make REST API calls to https://uaapps.dpi.wi.gov/EdFiWebApi/api/v2.0 

   1. Include a header “Authorization” with the value “Bearer” + Token

SERVICES

All of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/

PROFILES:

The following is a document which describes profiles in the Ed-Fi API: https://techdocs.ed-fi.org/display/ODSAPI20/API+Profiles

Wisconsin DPI’s main purpose with profiles is to limit the amount of data that a SIS vendor can send via the API. Wisconsin DPI only wants to collect certain elements as noted above. 

The default API Section dropdown value is Resources. A SIS can see the resources as restricted by the profile change the dropdown.

The API requires a content type be sent with a any HTTP action (ex. GET, POST). The content type is standardized, but lengthy. Public school clients trying to GET a student have to send the content type “application/vnd.ed-fi.student.widpi-profile-public.readable+json” which is highlighted in the image below. If the same SIS wants to POST a student school association they have to send the content type “application/vnd.ed-fi.studentschoolassociation.widpi-profile-public.writable+json”.

Note: the SIS still send the JSON to the same endpoints regardless of profiles.

CLAIMSETS:

The following document has more detail on claimsets: https://techdocs.ed-fi.org/display/ODSAPI20/Security+Configuration+Data+Stores

Wisconsin DPI restrict resources in the Ed-Fi API by linking a SIS key to a claimset. The claimset is linked to resources that a vendor can access and how they access the resource (authorization strategy). SIS vendors don’t need to know a lot about claimsets. If you GET/POST a resources outside the Wisconsin DPI domain you might get a error about a claimset. Wisconsin DPI can help confirm if the issue.

SAMPLES

All of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/

Note that samples may not contain all data required or conditionally required by Wisconsin DPI, but the samples should help you get started.  You may find JSON samples in the Data Domains & Endpoints area under each endpoint.

A couple of key notes:

1. studentUniqueId is the same as WISEid.  In production, Choice schools / districts will have to use the separate WI DPI WISEid system to obtain WISEids and upload or manual input to the SIS.  To get a set of WISEids for testing in the UAT ‘sandbox’, submit a request and a list of WISEids for testing will be provided.

2. SchoolId is created from the test Agency ID we create for your test school / district in the UAT ‘sandbox’ environment.  This is only to the UAT ‘sandbox’ environment. You will need to go to the WISEdata Ed-Fi Credential system to get the Agency ID (this is the same system you use to get your credentials (key and secret)) that can then be used to derive the schoolId (read this blog post). To get LEA and school ids for the production environment, take a look at this blog post for more information.

3. Retrieving students that you have already sent via a POST requires that you have sent a POST for the /studentSchoolAssociation related to that student.  You can only update and retrieve students in a school/district that relate to your credentials..

4. educationalOrganizationId in /studentAcademicRecord is the SchoolId.  It appears Ed-Fi kept this generic to allow for a district level or some other level of academic record other than school level.

https://wisconsindpi.atlassian.net/wiki/spaces/widpiedfi/pages/2294032/Data+Domains+Endpoints

SCHEMAS

All of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/

HTTP RESPONSES

RESPONSE CODES

A core tenet of RESTful APIs is embracing HTTP response codes to communicate status information. An API consumer should be able to inspect the  HTTP response code and understand the status of its request. The following  response codes are used when responding to requests. 

ERRORS

If an error occurs on the server, a 500 (Internal Server Error) code will be 

returned along with the addition of a “responseStatus” property in the response 

body, containing the error details. For example: 

{ 

 "responseStatus": { 

 "errorCode": "DataAccessException", 

 "message": "Unable to communicate with database", 

 "stackTrace": "…", 

 "errors": [] 

 } 

} 

WISEDATA ED-FI REST SDK

REQUIREMENTS

• Manually creating the Ed-Fi REST client: All of the steps to create the REST client can be found in the “readme” in the ED-FI REST SDK Repository: https://github.com/DoubleLinePartners/Ed-Fi-REST-SDK

The following is an outline of the steps need to manually create the client.

1. MACHINE SETUP

• Download the Java SDK

  • Download the Latest Java LTS Free

• Download SBT and install. Just under the "Installing sbt" header download the appropriate package for your operating system.  

• Ensure your environment can find javac. If you're on Windows set the PATH environment variable to find the jdk bin directory (e.g. C:\Program Files\Java\jdk1.7.0_51\bin).

• Download the ED-FI REST SDK Repository (look at the readme and its instructions) from the Double Line Partners GitHub 

  • https://github.com/DoubleLinePartners/Ed-Fi-REST-SDK

2. GENERATING THE REST CLIENT

• The rest of the instructions can be found in the “readme” in the DLP ED-FI REST SDK  

  • Open a command prompt to the ed-fi-rest-sdk folder and type "sbt assembly". This will build the Swagger codegen jar file.

  • Navigate to the folder where the jar file was created and run the command from the command prompt for each section in Swagger. This location is listed close to the end of the information log that is output during sbt generation.

  • Generate C# source files by running the generator once for each for each API section: descriptors, resources, and types.

  • java -jar sdk-generate.jar csharp --url https://{Domain root of API}/metadata/{section}/api-docs --baseDir {destination folder} --apiPackage {API namespace} --helperPackage {SDK namespace} --modelPackage {Model namespace}

  • Create a new class library in Visual Studio and include all of the generated files.

  • Use the Package Manager Console to install the RestSharp library (Install-Package Restsharp)