Create a csv file in Excel with the colums userprincipalname and password with the login username and the desired passwords and save as C:\temp\import.csv.

Open Powershell for Azure Active Directory.  After Connect-MsolService enter an administrator user name and password.

Connect-MsolService
$file = Import-CSV C:\temp\import.csv
ForEach ($record in $file){
Set-MsolUserPassword -userPrincipalName $record.userprincipalname –NewPassword $record.password -ForceChangePassword $False
} 

The output will show the passwords and any errors will be indicated in red.